Oct 30 15:35:16.311895 ip-10-0-140-110 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 15:35:16.311907 ip-10-0-140-110 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 15:35:16.311915 ip-10-0-140-110 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 15:35:16.312224 ip-10-0-140-110 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 15:35:26.438150 ip-10-0-140-110 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 15:35:26.438166 ip-10-0-140-110 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 66d660f4314343b4a6114512ad219a51 --
Oct 30 15:37:31.126716 ip-10-0-140-110 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 15:37:31.606205 ip-10-0-140-110 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 15:37:31.606205 ip-10-0-140-110 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 15:37:31.606205 ip-10-0-140-110 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 15:37:31.606205 ip-10-0-140-110 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 15:37:31.609374 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609267    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:37:31.609374 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609374    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:37:31.609374 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609380    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609383    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609386    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609390    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609394    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609397    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609399    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609402    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609405    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609407    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609410    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609413    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609415    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609418    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609421    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609423    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609426    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609428    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609431    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609433    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:37:31.609472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609437    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609439    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609442    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609454    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609457    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609459    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609462    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609465    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609467    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609493    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609498    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609502    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609505    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609508    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609511    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609515    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609518    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609520    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609523    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609525    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:37:31.609924 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609528    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609531    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609533    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609536    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609539    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609542    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609544    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609547    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609549    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609552    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609556    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609560    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609562    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609565    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609568    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609571    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609574    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609577    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609580    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609582    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:37:31.610409 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609585    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609587    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609590    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609593    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609595    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609598    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609600    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609603    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609605    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609608    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609611    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609614    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609618    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609622    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609625    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609628    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609631    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609634    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609637    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:37:31.610875 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609640    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609642    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609645    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609648    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609650    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609653    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609656    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609659    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609662    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609664    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609667    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609670    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609673    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609675    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609678    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609680    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609683    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609685    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609703    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:37:31.611328 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609706    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609710    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609712    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609715    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609718    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.609720    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610577    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610585    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610589    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610591    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610594    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610597    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610600    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610602    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610605    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610608    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610610    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610613    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610615    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610634    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:37:31.611822 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610638    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610640    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610643    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610645    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610649    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610651    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610655    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610657    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610660    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610663    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610665    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610668    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610671    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610673    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610676    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610678    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610681    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610683    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610686    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610689    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:37:31.612316 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610691    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610694    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610696    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610699    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610702    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610704    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610707    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610710    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610712    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610715    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610717    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610720    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610723    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610725    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610728    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610730    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610733    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610738    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610743    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:37:31.612876 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610746    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610749    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610752    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610755    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610758    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610760    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610763    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610765    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610768    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610770    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610773    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610775    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610778    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610780    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610783    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610785    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610788    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610791    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610794    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:37:31.613357 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610796    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610800    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610802    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610805    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610807    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610810    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610813    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610815    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610818    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610820    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610823    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610825    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610828    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610831    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610833    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610836    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610839    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610841    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610844    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:37:31.613814 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610846    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610849    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610851    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610854    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610856    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610861    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610864    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610868    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610871    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610873    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610876    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610879    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610881    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610884    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.610887    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611379    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611385    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611387    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611390    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:37:31.614267 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611393    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611395    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611398    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611401    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611403    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611406    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611409    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611411    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611414    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611417    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611420    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611422    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611424    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611427    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611429    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611432    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611434    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611437    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611443    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:37:31.614741 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611446    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611448    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611451    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611454    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611456    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611459    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611461    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611464    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611467    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611469    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611471    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611474    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611479    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611481    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611484    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611486    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611489    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611492    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611495    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:37:31.615208 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611498    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611501    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611504    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611506    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611508    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611511    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611513    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611517    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611519    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611522    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611524    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611527    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611530    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611533    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611536    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611538    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611541    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611543    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611546    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611548    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:37:31.615693 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611551    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611553    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611556    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611558    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611561    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611565    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611567    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611570    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611572    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611575    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611578    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611580    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611583    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611585    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611588    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611590    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611594    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611597    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611599    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:37:31.616167 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611602    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611604    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611607    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611609    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611612    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611616    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611618    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611621    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611623    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611626    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611629    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611631    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611633    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611636    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611639    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611641    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611644    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611647    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611650    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611653    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:37:31.616686 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611658    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611661    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611664    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611667    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.611677    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611816    2577 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611834    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611842    2577 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611848    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611852    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611856    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611860    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611865    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611868    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611871    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611875    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611878    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611882    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611886    2577 flags.go:64] FLAG: --cgroup-root=""
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611889    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611892    2577 flags.go:64] FLAG: --client-ca-file=""
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611895    2577 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611898    2577 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 15:37:31.617152 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611903    2577 flags.go:64] FLAG: --cluster-domain=""
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611906    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611909    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611912    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611916    2577 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611920    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611923    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611927    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611931    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611934    2577 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611937    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611940    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611943    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611946    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611951    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611955    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611958    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611962    2577 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611965    2577 flags.go:64] FLAG: --enable-server="true"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611968    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611974    2577 flags.go:64] FLAG: --event-burst="100"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611977    2577 flags.go:64] FLAG: --event-qps="50"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611980    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611984    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 15:37:31.617719 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611987    2577 flags.go:64] FLAG: --eviction-hard=""
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611991    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611995    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.611998    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612001    2577 flags.go:64] FLAG: --eviction-soft=""
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612004    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612007    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612010    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612013    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612016    2577 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612019    2577 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612022    2577 flags.go:64] FLAG: --feature-gates=""
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612029    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612032    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612035    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612038    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612043    2577 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612046    2577 flags.go:64] FLAG: --help="false"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612049    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-140-110.ec2.internal"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612052    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612055    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612058    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612063    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 15:37:31.618416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612069    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612072    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612075    2577 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612079    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612081    2577 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612085    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612088    2577 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612091    2577 flags.go:64] FLAG: --kube-reserved=""
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612094    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612097    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612100    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612104    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612106    2577 flags.go:64] FLAG: --lock-file=""
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612109    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612112    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612115    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612121    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612124    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612127    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612130    2577 flags.go:64] FLAG: --logging-format="text"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612133    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612137    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612140    2577 flags.go:64] FLAG: --manifest-url=""
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612143    2577 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612148    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 15:37:31.619025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612151    2577 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612156    2577 flags.go:64] FLAG: --max-pods="110"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612159    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612162    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612165    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612168    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612171    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612176    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612179    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612186    2577 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612189    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612193    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612196    2577 flags.go:64] FLAG: --pod-cidr=""
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612199    2577 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612202    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612206    2577 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612209    2577 flags.go:64] FLAG: --port="10250"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612212    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612215    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0609ecc1757f97782"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612218    2577 flags.go:64] FLAG: --qos-reserved=""
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612221    2577 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612224    2577 flags.go:64] FLAG: --register-node="true"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612227    2577 flags.go:64] FLAG: --register-with-taints=""
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612231    2577 flags.go:64] FLAG: --registry-burst="10"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612234    2577 flags.go:64] FLAG: --registry-qps="5"
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612237    2577 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 15:37:31.620015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612240    2577 flags.go:64] FLAG: --reserved-memory=""
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612244    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612260    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612265    2577 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612268    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612272    2577 flags.go:64] FLAG: --runonce="false"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612275    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612278    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612282    2577 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612286    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612289    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612292    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612295    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612302    2577 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612305    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612308    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612311    2577 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612314    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612318    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612321    2577 flags.go:64] FLAG: --system-cgroups=""
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612324    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612330    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612333    2577 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612336    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612340    2577 flags.go:64] FLAG: --tls-min-version=""
Oct 30 15:37:31.620849 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612343    2577 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612358    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612362    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612365    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612368    2577 flags.go:64] FLAG: --v="2"
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612373    2577 flags.go:64] FLAG: --version="false"
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612377    2577 flags.go:64] FLAG: --vmodule=""
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612381    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612384    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612581    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612586    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612589    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612593    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612596    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612599    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612601    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612610    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612614    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612617    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612620    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612625    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612628    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:37:31.621469 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612631    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612633    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612636    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612639    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612641    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612644    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612646    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612649    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612652    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612654    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612657    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612659    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612662    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612665    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612667    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612670    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612672    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612675    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612677    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612680    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:37:31.621994 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612682    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612685    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612688    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612690    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612692    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612695    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612698    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612706    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612709    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612712    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612717    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612720    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612722    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612725    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612727    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612730    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612733    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612735    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612738    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:37:31.622534 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612740    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612743    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612745    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612748    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612751    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612753    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612756    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612758    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612761    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612763    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612766    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612768    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612771    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612773    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612776    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612779    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612781    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612784    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612786    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612789    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:37:31.622986 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612791    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612800    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612803    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612808    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612810    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612813    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612815    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612818    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612821    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612823    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612826    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612828    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612831    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612833    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612836    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612838    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612842    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612846    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612848    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612851    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:37:31.623472 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612854    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612857    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612860    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612863    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612865    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612868    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612872    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612875    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612878    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612881    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612884    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612887    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612889    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612892    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.612927    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 15:37:31.623951 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612973    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612979    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612983    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612986    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612989    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612991    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612994    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.612997    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613000    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613002    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613005    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613007    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613010    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613012    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613015    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613018    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613020    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613023    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613025    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613028    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:37:31.624313 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613030    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613033    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613036    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613038    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613040    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613043    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613046    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613048    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613051    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613054    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613056    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613059    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613067    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613072    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613075    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613077    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613080    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613082    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613085    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:37:31.624810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613087    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613090    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613092    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613095    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613098    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613100    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613103    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613105    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613107    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613110    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613113    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613115    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613118    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613121    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613123    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613126    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613128    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613131    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613133    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:37:31.625260 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613136    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613138    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613142    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613144    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613147    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613149    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613152    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613162    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613165    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613167    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613170    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613173    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613175    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613178    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613181    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613183    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613186    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613188    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613191    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613193    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:37:31.625790 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613196    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613198    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613201    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613203    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613207    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613210    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613213    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613216    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613219    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613222    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613224    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613227    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613230    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613232    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613236    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613240    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613243    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613247    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613250    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:37:31.626414 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613254    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613262    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613265    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613268    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613270    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613273    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613275    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613278    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.613280    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.621316    2577 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.621332    2577 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.622317    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.622324    2577 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622399    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622405    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622409    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622412    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622414    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622417    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:37:31.626921 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622420    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622423    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622426    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622428    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622431    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622433    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622436    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622439    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622441    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622444    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622447    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622454    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622457    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622460    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622463    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622465    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622468    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622470    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622473    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:37:31.627394 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622476    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622479    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622482    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622484    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622487    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622490    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622493    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622495    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622498    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622500    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622503    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622505    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622508    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622510    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622513    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622516    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622518    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622521    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622523    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:37:31.627928 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622526    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622528    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622531    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622534    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622537    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622540    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622543    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622546    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622548    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622552    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622554    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622557    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622559    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622562    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622564    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622567    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622570    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622572    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622575    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622577    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:37:31.628460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622580    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622582    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622585    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622587    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622590    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622592    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622595    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622597    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622600    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622604    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622608    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622611    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622613    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622616    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622618    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622627    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622630    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622633    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622635    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:37:31.628957 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622638    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622640    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622643    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622646    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622648    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622651    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622654    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622656    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622659    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622661    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622664    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622666    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622669    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622672    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622674    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622677    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622679    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622682    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622684    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:37:31.629460 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622687    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622691    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622694    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622698    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.622726    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622881    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622888    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622891    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622893    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622896    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622900    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622903    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622906    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622908    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622912    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:37:31.629903 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622917    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622920    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622923    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622926    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622929    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622931    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622934    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622936    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622939    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622942    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622944    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622947    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622949    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622952    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622954    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622957    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622960    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622962    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622965    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622967    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:37:31.630254 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622970    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622972    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622975    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622978    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622981    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622984    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622986    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622989    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622993    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622996    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.622999    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623001    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623004    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623006    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623009    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623012    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623014    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623017    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623019    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623022    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:37:31.630770 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623025    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623027    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623030    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623032    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623035    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623039    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623043    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623046    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623049    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623052    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623054    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623057    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623059    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623062    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623065    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623067    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623070    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623072    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623075    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:37:31.631318 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623077    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623082    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623085    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623087    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623090    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623093    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623096    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623098    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623101    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623104    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623106    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623108    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623111    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623114    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623116    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623119    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623121    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623124    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623126    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:37:31.631841 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623129    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623132    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623134    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623137    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623141    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623143    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623146    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623148    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623151    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623154    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623156    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623159    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623161    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623164    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623168    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623170    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623173    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:37:31.632478 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623176    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.623198    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623231    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623235    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623239    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623243    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623246    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623249    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623251    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623254    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623257    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623259    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623262    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623265    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623267    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623270    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:37:31.633143 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623273    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623275    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623278    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623280    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623285    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623287    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623290    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623293    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623295    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623298    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623301    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623303    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623306    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623310    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623312    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623315    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623318    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623321    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623323    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:37:31.633576 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623326    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623328    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623331    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623334    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623336    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623339    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623341    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623354    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623357    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623360    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623362    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623365    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623367    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623370    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623373    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623375    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623378    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623381    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623384    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:37:31.634108 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623387    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623389    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623392    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623395    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623397    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623400    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623402    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623407    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623410    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623412    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623416    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623420    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623423    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623426    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623428    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623431    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623434    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623436    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623439    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:37:31.634810 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623441    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623444    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623447    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623450    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623452    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623455    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623458    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623460    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623463    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623466    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623468    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623472    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623475    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623478    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623481    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623483    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623486    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623489    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623492    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623494    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:37:31.635382 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623497    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623500    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623503    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623506    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623508    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623511    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623515    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623518    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623521    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623523    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623526    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623528    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623531    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623533    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:31.623536    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.623588    2577 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.626460    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.627570    2577 server.go:1014] "Starting client certificate rotation"
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.627684    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 15:37:31.635916 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.628931    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 15:37:31.636377 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.631340    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 15:37:31.636377 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.635106    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 15:37:31.645925 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.645908    2577 log.go:25] "Validated CRI v1 runtime API"
Oct 30 15:37:31.648933 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.648918    2577 log.go:25] "Validated CRI v1 image API"
Oct 30 15:37:31.649759 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.649738    2577 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 15:37:31.652426 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.652403    2577 fs.go:135] Filesystem UUIDs: map[2df379d2-e4bf-4b2b-8874-576ed76ba068:/dev/nvme0n1p3 388aa26c-e611-43aa-9553-ffac4cc82e87:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 30 15:37:31.652489 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.652426    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 15:37:31.657943 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.657809    2577 manager.go:217] Machine: {Timestamp:2026-10-30 15:37:31.656284594 +0000 UTC m=+0.390642948 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099715 MemoryCapacity:33164328960 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ec75f85b724e63ecf0d8bb54d5c32 SystemUUID:ec2ec75f-85b7-24e6-3ecf-0d8bb54d5c32 BootID:66d660f4-3143-43b4-a611-4512ad219a51 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d8:f0:f3:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d8:f0:f3:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:8d:bd:7f:57:1f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164328960 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 15:37:31.657943 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.657942    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 15:37:31.658049 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.658018    2577 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 15:37:31.662233 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.662209    2577 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 15:37:31.662379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.662235    2577 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-110.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 15:37:31.662426 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.662381    2577 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 15:37:31.662426 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.662389    2577 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 15:37:31.662480 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.662426    2577 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 15:37:31.662480 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.662442    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 15:37:31.663285 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.663276    2577 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 15:37:31.668983 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.668969    2577 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 15:37:31.669119 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.669109    2577 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 15:37:31.669923 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.669911    2577 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 15:37:31.669954 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.669931    2577 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 15:37:31.669954 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.669945    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 15:37:31.670039 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.669957    2577 kubelet.go:405] "Adding apiserver pod source"
Oct 30 15:37:31.670039 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.669969    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 15:37:31.672695 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.672645    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 15:37:31.675612 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.675593    2577 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 15:37:31.681471 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.681455    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 15:37:31.682402 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.682389    2577 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 15:37:31.682454 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.682420    2577 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 15:37:31.682577 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.682564    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 15:37:31.682610 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.682587    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 15:37:31.682610 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.682597    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 15:37:31.682610 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.682606    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 15:37:31.682694 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.682613    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 15:37:31.682694 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.682622    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 15:37:31.683096 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.683086    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 15:37:31.683123 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.683098    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 15:37:31.683123 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.683105    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 15:37:31.683123 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.683111    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 15:37:31.683123 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.683121    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 15:37:31.683269 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.683129    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 15:37:31.684460 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.684449    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 15:37:31.684494 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.684463    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 15:37:31.688956 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.688939    2577 server.go:1263] "Started kubelet"
Oct 30 15:37:31.689038 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.689005    2577 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 15:37:31.689764 ip-10-0-140-110 systemd[1]: Started Kubernetes Kubelet.
Oct 30 15:37:31.690227 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.689745    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 15:37:31.690227 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.689818    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 15:37:31.691175 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.691157    2577 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 15:37:31.691500 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.691477    2577 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 15:37:31.693896 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:31.693875    2577 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 15:37:31.716503 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.716467    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-110.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 15:37:31.717278 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:31.716471    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-110.ec2.internal.18e358be771b7509  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-110.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-110.ec2.internal,},FirstTimestamp:2026-10-30 15:37:31.688908041 +0000 UTC m=+0.423266394,LastTimestamp:2026-10-30 15:37:31.688908041 +0000 UTC m=+0.423266394,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-110.ec2.internal,}"
Oct 30 15:37:31.725474 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.725451    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 15:37:31.725575 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.725506    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lnqw9"
Oct 30 15:37:31.737569 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.737549    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lnqw9"
Oct 30 15:37:31.738236 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:31.738217    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 15:37:31.771555 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:31.771521    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 15:37:31.824673 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.824644    2577 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 15:37:31.824673 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.824660    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 15:37:31.824862 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:31.824728    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-110.ec2.internal\" not found"
Oct 30 15:37:31.825702 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.825683    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 15:37:31.825801 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.825735    2577 factory.go:55] Registering systemd factory
Oct 30 15:37:31.825801 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.825753    2577 factory.go:223] Registration of the systemd container factory successfully
Oct 30 15:37:31.825873 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.825805    2577 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 15:37:31.825873 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.825816    2577 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 15:37:31.825958 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.825944    2577 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 15:37:31.826007 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.825959    2577 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 15:37:31.826505 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.826477    2577 factory.go:153] Registering CRI-O factory
Oct 30 15:37:31.826505 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.826503    2577 factory.go:223] Registration of the crio container factory successfully
Oct 30 15:37:31.826634 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.826581    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 15:37:31.826634 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.826605    2577 factory.go:103] Registering Raw factory
Oct 30 15:37:31.826634 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.826625    2577 manager.go:1199] Started watching for new ooms in manager
Oct 30 15:37:31.828837 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.827905    2577 manager.go:319] Starting recovery of all containers
Oct 30 15:37:31.838230 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.838209    2577 manager.go:324] Recovery completed
Oct 30 15:37:31.839684 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:31.839644    2577 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 15:37:31.842547 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.842530    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 15:37:31.844105 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.844086    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 15:37:31.844177 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.844123    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 15:37:31.844177 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.844137    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientPID"
Oct 30 15:37:31.844790 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.844775    2577 cpu_manager.go:226] "Starting" policy="none"
Oct 30 15:37:31.844790 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.844788    2577 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 15:37:31.844908 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.844811    2577 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 15:37:31.844908 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.844838    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 15:37:31.845547 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.845533    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 15:37:31.845601 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.845559    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 15:37:31.845601 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.845569    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientPID"
Oct 30 15:37:31.846866 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.846851    2577 policy_none.go:50] "Start"
Oct 30 15:37:31.846933 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.846869    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 15:37:31.846933 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.846886    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 15:37:31.848857 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.848842    2577 policy_none.go:44] "Start"
Oct 30 15:37:31.881966 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.881830    2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:37:31.883933 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.883905    2577 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 15:37:31.886214 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.886197    2577 manager.go:341] "Starting Device Plugin manager"
Oct 30 15:37:31.886301 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:31.886226    2577 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 15:37:31.886301 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.886235    2577 server.go:87] "Starting device plugin registration server"
Oct 30 15:37:31.886536 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.886520    2577 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 15:37:31.886646 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.886536    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 15:37:31.886646 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.886611    2577 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 15:37:31.886745 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.886688    2577 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 15:37:31.886745 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.886695    2577 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 15:37:31.887281 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:31.887254    2577 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 15:37:31.887356 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:31.887303    2577 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-110.ec2.internal\" not found"
Oct 30 15:37:31.951535 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.951493    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 15:37:31.953444 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.952700    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 15:37:31.953444 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.952721    2577 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 15:37:31.953444 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.952739    2577 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 15:37:31.953444 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:31.952774    2577 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 15:37:31.969798 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.969778    2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:37:31.986908 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.986887    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 15:37:31.987792 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.987777    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 15:37:31.987865 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.987804    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 15:37:31.987865 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.987820    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientPID"
Oct 30 15:37:31.987865 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:31.987847    2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.009439 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.009421    2577 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.009494 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.009444    2577 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-110.ec2.internal\": node \"ip-10-0-140-110.ec2.internal\" not found"
Oct 30 15:37:32.053542 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.053500    2577 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"]
Oct 30 15:37:32.053709 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.053583    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 15:37:32.054553 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.054534    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 15:37:32.054657 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.054568    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 15:37:32.054657 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.054581    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientPID"
Oct 30 15:37:32.056832 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.056817    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 15:37:32.056996 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.056981    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.057042 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.057010    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 15:37:32.057534 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.057513    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 15:37:32.057619 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.057514    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 15:37:32.057619 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.057560    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 15:37:32.057619 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.057570    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientPID"
Oct 30 15:37:32.057619 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.057545    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 15:37:32.057619 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.057615    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientPID"
Oct 30 15:37:32.059598 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.059586    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.059649 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.059611    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 15:37:32.060295 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.060272    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 15:37:32.060385 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.060300    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 15:37:32.060385 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.060314    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientPID"
Oct 30 15:37:32.068211 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.068194    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-110.ec2.internal\" not found"
Oct 30 15:37:32.081143 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.081121    2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-110.ec2.internal\" not found" node="ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.083068 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.083051    2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-110.ec2.internal\" not found" node="ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.127680 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.127580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.127680 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.127639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.127811 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.127715    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50080b7deadb8e30dee000b7f0bbadfa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-110.ec2.internal\" (UID: \"50080b7deadb8e30dee000b7f0bbadfa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.168804 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.168776    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-110.ec2.internal\" not found"
Oct 30 15:37:32.227942 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.227862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.228012 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.227969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50080b7deadb8e30dee000b7f0bbadfa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-110.ec2.internal\" (UID: \"50080b7deadb8e30dee000b7f0bbadfa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.228012 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.227989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.228012 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.227928    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.228115 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.228027    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.228115 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.228048    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50080b7deadb8e30dee000b7f0bbadfa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-110.ec2.internal\" (UID: \"50080b7deadb8e30dee000b7f0bbadfa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.269604 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.269580    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-110.ec2.internal\" not found"
Oct 30 15:37:32.370209 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.370176    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-110.ec2.internal\" not found"
Oct 30 15:37:32.384954 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.384894    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.386088 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.386068    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.473582 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.473548    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-110.ec2.internal\" not found"
Oct 30 15:37:32.571316 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.571289    2577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:37:32.628172 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.628121    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 15:37:32.628754 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.628179    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.628754 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.628287    2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 15:37:32.628754 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.628332    2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 15:37:32.628754 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.628367    2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 15:37:32.672066 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.671982    2577 apiserver.go:52] "Watching apiserver"
Oct 30 15:37:32.715937 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.715899    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 15:37:32.716115 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.715973    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:37:32.716115 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.716040    2577 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 15:37:32.716348 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.716332    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-zxvbz","kube-system/global-pull-secret-syncer-9l9gb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj","openshift-multus/multus-additional-cni-plugins-9xhq8","openshift-ovn-kubernetes/ovnkube-node-sg7k4","kube-system/konnectivity-agent-trlcx","openshift-multus/multus-lw4vm","openshift-network-diagnostics/network-check-target-ttqnr","openshift-image-registry/node-ca-xrrf9","openshift-multus/network-metrics-daemon-fqdrs","openshift-cluster-node-tuning-operator/tuned-4wfrg","openshift-dns/node-resolver-ngvd2"]
Oct 30 15:37:32.718945 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.718928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.721052 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.721023    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.723195 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.723173    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:32.725704 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.725462    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:32.725704 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.725532    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:32.727766 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.727747    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.729970 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.729955    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.730878 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.730858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-sys-fs\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.730955 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.730883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-dbus\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:32.730955 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.730902    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-registration-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.730955 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.730920    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysconfig\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731093 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.730966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-run\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731093 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.730993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-lib-modules\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731093 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-var-lib-kubelet\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731093 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731025    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-host\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731093 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e724db28-b13f-4bfa-89ba-98eb630e4241-iptables-alerter-script\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:32.731093 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.731346 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-kubernetes\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731346 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-sys\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731346 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-tmp\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731346 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tgcb\" (UniqueName: \"kubernetes.io/projected/e894be05-f132-4b29-a974-2ddf29c3e991-kube-api-access-9tgcb\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731346 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-kubelet-config\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:32.731346 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.731346 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.731682 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731682 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-conf\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.731682 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4lxq\" (UniqueName: \"kubernetes.io/projected/e724db28-b13f-4bfa-89ba-98eb630e4241-kube-api-access-h4lxq\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:32.731682 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.731682 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.731682 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-socket-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.731682 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-device-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.731682 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e724db28-b13f-4bfa-89ba-98eb630e4241-host-slash\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:32.731682 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.732037 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq6rt\" (UniqueName: \"kubernetes.io/projected/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kube-api-access-vq6rt\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.732037 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-systemd\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.732037 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xmmj\" (UniqueName: \"kubernetes.io/projected/c1938ddf-e712-417e-9741-8723017995b5-kube-api-access-2xmmj\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.732037 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-modprobe-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.732037 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-etc-tuned\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.732037 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:32.732037 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731907    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-system-cni-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.732037 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.731943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-cnibin\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.732324 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.732311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:32.734624 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.734600    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 15:37:32.734624 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.734621    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 15:37:32.734778 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.734636    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-khtmm\""
Oct 30 15:37:32.734778 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.734686    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 15:37:32.734778 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.734724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.736205 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.736179    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-84s7k\""
Oct 30 15:37:32.736311 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.736279    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 15:37:32.736375 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.736357    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 15:37:32.738128 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.738107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:32.738230 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.738189    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:32.738491 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.738471    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 15:37:32.740501 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.740485    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:32.743240 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.743224    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:32.743332 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.743287    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:32.745540 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.745521    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"]
Oct 30 15:37:32.745611 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.745549    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"]
Oct 30 15:37:32.745611 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.745606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:32.825541 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.825507    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 15:37:32.826178 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.826124    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 15:32:32 +0000 UTC" deadline="2028-03-31 16:24:08.302836251 +0000 UTC"
Oct 30 15:37:32.826178 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.826176    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12432h46m35.476663798s"
Oct 30 15:37:32.827170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.827150    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 15:37:32.832196 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-registration-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.832268 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-var-lib-kubelet\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.832268 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-host\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.832344 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832282    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-slash\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.832344 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832297    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-registration-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.832344 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e724db28-b13f-4bfa-89ba-98eb630e4241-iptables-alerter-script\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:32.832344 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832303    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-host\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.832519 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832341    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-var-lib-kubelet\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.832519 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832343    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-ovn\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.832519 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.832519 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-kubernetes\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.832519 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832482    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.832519 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832490    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-kubernetes\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.832519 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832494    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.832876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.832876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832583    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-etc-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.832876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-systemd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.832876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/587ce512-7067-42a2-9cf1-2babd4feae72-agent-certs\") pod \"konnectivity-agent-trlcx\" (UID: \"587ce512-7067-42a2-9cf1-2babd4feae72\") " pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:32.832876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-cnibin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.832876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-netns\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.832876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832763    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-multus\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.832876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-socket-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.832876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832832    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-device-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.832876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832930    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-socket-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-multus-daemon-config\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.832989    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-device-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xmmj\" (UniqueName: \"kubernetes.io/projected/c1938ddf-e712-417e-9741-8723017995b5-kube-api-access-2xmmj\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833130    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-bin\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-config\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-host\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-hostroot\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-modprobe-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:32.833379 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-system-cni-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-cnibin\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833417    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-system-cni-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833419    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-sys-fs\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833455    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-sys-fs\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-systemd-units\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833479    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-modprobe-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833484    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-serviceca\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833493    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-cnibin\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrsng\" (UniqueName: \"kubernetes.io/projected/4cc7d376-7849-4d50-8c0f-3128e474b54c-kube-api-access-nrsng\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-system-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.833535    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysconfig\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-run\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-lib-modules\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.833676    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret podName:75385577-f877-463f-a0fc-c9e2b844e840 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:33.333645101 +0000 UTC m=+2.068003465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret") pod "global-pull-secret-syncer-9l9gb" (UID: "75385577-f877-463f-a0fc-c9e2b844e840") : object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833698    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysconfig\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.834106 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-env-overrides\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833745    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-lib-modules\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833760    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-etc-kubernetes\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-netns\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833805    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-run\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/587ce512-7067-42a2-9cf1-2babd4feae72-konnectivity-ca\") pod \"konnectivity-agent-trlcx\" (UID: \"587ce512-7067-42a2-9cf1-2babd4feae72\") " pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-sys\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833965    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-tmp\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.833991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tgcb\" (UniqueName: \"kubernetes.io/projected/e894be05-f132-4b29-a974-2ddf29c3e991-kube-api-access-9tgcb\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834034    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-sys\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-kubelet-config\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834171    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-kubelet-config\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834188    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-kubelet\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovn-node-metrics-cert\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.835025 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-conf\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4lxq\" (UniqueName: \"kubernetes.io/projected/e724db28-b13f-4bfa-89ba-98eb630e4241-kube-api-access-h4lxq\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834356    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-679w9\" (UniqueName: \"kubernetes.io/projected/89518dc7-2a05-4db9-b4f4-0662fd34d01b-kube-api-access-679w9\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834382    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834397    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmmnk\" (UniqueName: \"kubernetes.io/projected/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-kube-api-access-zmmnk\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834406    2577 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc7d376-7849-4d50-8c0f-3128e474b54c-hosts-file\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834414    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834456    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-conf\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834473    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-k8s-cni-cncf-io\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e724db28-b13f-4bfa-89ba-98eb630e4241-host-slash\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67l68\" (UniqueName: \"kubernetes.io/projected/eea4d604-1949-4ba4-964d-e57f74ce431a-kube-api-access-67l68\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834570    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-log-socket\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834583    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e724db28-b13f-4bfa-89ba-98eb630e4241-host-slash\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834611    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-netd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc7d376-7849-4d50-8c0f-3128e474b54c-tmp-dir\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:32.835672 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834699    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-socket-dir-parent\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-conf-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qrm4\" (UniqueName: \"kubernetes.io/projected/2d430d20-3538-482d-8588-a073be77dbd5-kube-api-access-5qrm4\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq6rt\" (UniqueName: \"kubernetes.io/projected/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kube-api-access-vq6rt\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834850    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-systemd\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-node-log\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-script-lib\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834892    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-systemd\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834950    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-etc-tuned\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.834999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.835037    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.835102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-cni-binary-copy\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.835138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-bin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.835161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-kubelet\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.835235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-dbus\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.835275    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-var-lib-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.836322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.835303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-multus-certs\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.836876 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.835394    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-dbus\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:32.838444 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.838399    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-tmp\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.838444 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.838414    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-etc-tuned\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.844015 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.843996    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 15:37:32.844142 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.844124    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 15:37:32.844590 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.844570    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 15:37:32.844681 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.844595    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 15:37:32.844681 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.844672    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 15:37:32.844782 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.844711    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ck89s\""
Oct 30 15:37:32.845121 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.845078    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 15:37:32.845274 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.845254    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 15:37:32.845492 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.845146    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 15:37:32.845724 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.845695    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.845834 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.845816    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 15:37:32.845942 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.845920    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 15:37:32.845942 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.845935    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 15:37:32.846246 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.846226    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 15:37:32.846416 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.846395    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 15:37:32.846557 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.846539    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ftvkm\""
Oct 30 15:37:32.846638 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.846623    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 15:37:32.846776 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.846756    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-xbwc7\""
Oct 30 15:37:32.846857 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.846832    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fmlw7\""
Oct 30 15:37:32.846973 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.846956    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 15:37:32.847284 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.847266    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5dknm\""
Oct 30 15:37:32.847436 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.847272    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 15:37:32.853933 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.853910    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.854104 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.853950    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.854104 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.853963    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e724db28-b13f-4bfa-89ba-98eb630e4241-iptables-alerter-script\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:32.856741 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.856716    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 15:37:32.856835 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.856730    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 15:37:32.857204 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.857193    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8vjmf\""
Oct 30 15:37:32.857561 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.857543    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 15:37:32.857626 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.857583    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 15:37:32.857626 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.857589    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 15:37:32.857764 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.857750    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 15:37:32.857817 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.857753    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mw526\""
Oct 30 15:37:32.865790 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.865609    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xmmj\" (UniqueName: \"kubernetes.io/projected/c1938ddf-e712-417e-9741-8723017995b5-kube-api-access-2xmmj\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:32.866565 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.866536    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tgcb\" (UniqueName: \"kubernetes.io/projected/e894be05-f132-4b29-a974-2ddf29c3e991-kube-api-access-9tgcb\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:32.868905 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.868882    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4lxq\" (UniqueName: \"kubernetes.io/projected/e724db28-b13f-4bfa-89ba-98eb630e4241-kube-api-access-h4lxq\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:32.874993 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.874968    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq6rt\" (UniqueName: \"kubernetes.io/projected/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kube-api-access-vq6rt\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:32.905582 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.905553    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 15:37:32.935926 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.935849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:32.935926 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.935908    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679w9\" (UniqueName: \"kubernetes.io/projected/89518dc7-2a05-4db9-b4f4-0662fd34d01b-kube-api-access-679w9\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936120 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.935928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmmnk\" (UniqueName: \"kubernetes.io/projected/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-kube-api-access-zmmnk\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:32.936120 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.935947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc7d376-7849-4d50-8c0f-3128e474b54c-hosts-file\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:32.936120 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.935963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-k8s-cni-cncf-io\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936120 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.935986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67l68\" (UniqueName: \"kubernetes.io/projected/eea4d604-1949-4ba4-964d-e57f74ce431a-kube-api-access-67l68\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:32.936120 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-log-socket\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936120 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936060    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-k8s-cni-cncf-io\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936120 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936069    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc7d376-7849-4d50-8c0f-3128e474b54c-hosts-file\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:32.936120 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.936074    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936120    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-log-socket\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936143    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-netd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936104    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-netd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.936175    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs podName:eea4d604-1949-4ba4-964d-e57f74ce431a nodeName:}" failed. No retries permitted until 2026-10-30 15:37:33.436154868 +0000 UTC m=+2.170513224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs") pod "network-metrics-daemon-fqdrs" (UID: "eea4d604-1949-4ba4-964d-e57f74ce431a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc7d376-7849-4d50-8c0f-3128e474b54c-tmp-dir\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-socket-dir-parent\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-conf-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936417    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-conf-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936447    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-socket-dir-parent\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qrm4\" (UniqueName: \"kubernetes.io/projected/2d430d20-3538-482d-8588-a073be77dbd5-kube-api-access-5qrm4\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-node-log\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936533 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-script-lib\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-cni-binary-copy\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936641    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-node-log\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-bin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936644    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936681    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-kubelet\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-var-lib-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-multus-certs\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936708    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936747    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-kubelet\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-slash\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936789    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-multus-certs\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936789    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-var-lib-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936736    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-bin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.936991 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-ovn\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936828    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-slash\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-etc-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936840    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-ovn\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936849    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-etc-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-systemd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/587ce512-7067-42a2-9cf1-2babd4feae72-agent-certs\") pod \"konnectivity-agent-trlcx\" (UID: \"587ce512-7067-42a2-9cf1-2babd4feae72\") " pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-cnibin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936929    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-systemd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-netns\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.936982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-multus\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937008    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-cnibin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937055    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-multus\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937105    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-multus-daemon-config\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-bin\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.937791 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-config\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937167    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-host\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937237    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-bin\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-hostroot\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-systemd-units\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937301    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-host\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-serviceca\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrsng\" (UniqueName: \"kubernetes.io/projected/4cc7d376-7849-4d50-8c0f-3128e474b54c-kube-api-access-nrsng\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-system-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937349    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-hostroot\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-env-overrides\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-etc-kubernetes\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937408    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-netns\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937402    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-systemd-units\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-netns\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/587ce512-7067-42a2-9cf1-2babd4feae72-konnectivity-ca\") pod \"konnectivity-agent-trlcx\" (UID: \"587ce512-7067-42a2-9cf1-2babd4feae72\") " pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937476    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-system-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.938663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-kubelet\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937520    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-etc-kubernetes\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937522    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-kubelet\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937544    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-netns\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.937585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovn-node-metrics-cert\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.938134    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc7d376-7849-4d50-8c0f-3128e474b54c-tmp-dir\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.938206    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-cni-binary-copy\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.938292    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/587ce512-7067-42a2-9cf1-2babd4feae72-konnectivity-ca\") pod \"konnectivity-agent-trlcx\" (UID: \"587ce512-7067-42a2-9cf1-2babd4feae72\") " pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.938360    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-serviceca\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.938391    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-multus-daemon-config\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.938548    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-config\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.938558    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-script-lib\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.939170 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.938747    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-env-overrides\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.940090 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.940073    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/587ce512-7067-42a2-9cf1-2babd4feae72-agent-certs\") pod \"konnectivity-agent-trlcx\" (UID: \"587ce512-7067-42a2-9cf1-2babd4feae72\") " pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:32.940219 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.940202    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovn-node-metrics-cert\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.960607 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.960576    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-679w9\" (UniqueName: \"kubernetes.io/projected/89518dc7-2a05-4db9-b4f4-0662fd34d01b-kube-api-access-679w9\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:32.963115 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.963090    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 15:37:32.963115 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.963114    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 15:37:32.963296 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.963125    2577 projected.go:196] Error preparing data for projected volume kube-api-access-njltq for pod openshift-network-diagnostics/network-check-target-ttqnr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:32.963296 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:32.963184    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq podName:746ec817-962d-4cb7-9767-0be957f2c533 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:33.463169394 +0000 UTC m=+2.197527750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-njltq" (UniqueName: "kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq") pod "network-check-target-ttqnr" (UID: "746ec817-962d-4cb7-9767-0be957f2c533") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:32.965107 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.965081    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmmnk\" (UniqueName: \"kubernetes.io/projected/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-kube-api-access-zmmnk\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:32.965378 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.965354    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67l68\" (UniqueName: \"kubernetes.io/projected/eea4d604-1949-4ba4-964d-e57f74ce431a-kube-api-access-67l68\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:32.965689 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.965657    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrsng\" (UniqueName: \"kubernetes.io/projected/4cc7d376-7849-4d50-8c0f-3128e474b54c-kube-api-access-nrsng\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:32.967005 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.966973    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qrm4\" (UniqueName: \"kubernetes.io/projected/2d430d20-3538-482d-8588-a073be77dbd5-kube-api-access-5qrm4\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:32.990011 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:32.989980    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cd74d1416e17adb1d88ce1b2d585255.slice/crio-bf1499191d4923a8cb698ff7331daa7d998e88477d80296871135bdfbb9a65c3 WatchSource:0}: Error finding container bf1499191d4923a8cb698ff7331daa7d998e88477d80296871135bdfbb9a65c3: Status 404 returned error can't find the container with id bf1499191d4923a8cb698ff7331daa7d998e88477d80296871135bdfbb9a65c3
Oct 30 15:37:32.991917 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.991895    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ljpzg"
Oct 30 15:37:32.993087 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:32.993062    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50080b7deadb8e30dee000b7f0bbadfa.slice/crio-e416e74a75496c398348f99d57a4bf534ea1e1a0a8672a916e4ac8ade6b40f2e WatchSource:0}: Error finding container e416e74a75496c398348f99d57a4bf534ea1e1a0a8672a916e4ac8ade6b40f2e: Status 404 returned error can't find the container with id e416e74a75496c398348f99d57a4bf534ea1e1a0a8672a916e4ac8ade6b40f2e
Oct 30 15:37:32.995285 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:32.995265    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 15:37:33.005359 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.005335    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ljpzg"
Oct 30 15:37:33.049541 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.049505    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:37:33.058272 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:33.058247    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74a78648_f7a8_4363_9524_8bbc4b6a14a3.slice/crio-162edece99bb29c7a00db9fdadf2c50e8bdbc11b52679c503295942edb970da0 WatchSource:0}: Error finding container 162edece99bb29c7a00db9fdadf2c50e8bdbc11b52679c503295942edb970da0: Status 404 returned error can't find the container with id 162edece99bb29c7a00db9fdadf2c50e8bdbc11b52679c503295942edb970da0
Oct 30 15:37:33.061722 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.061698    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:37:33.068375 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.068266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:37:33.070698 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:33.070666    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode894be05_f132_4b29_a974_2ddf29c3e991.slice/crio-5256d5c98bcb08f9b3067d3260527b694c411ece063e65e4fd94954336887d01 WatchSource:0}: Error finding container 5256d5c98bcb08f9b3067d3260527b694c411ece063e65e4fd94954336887d01: Status 404 returned error can't find the container with id 5256d5c98bcb08f9b3067d3260527b694c411ece063e65e4fd94954336887d01
Oct 30 15:37:33.077268 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:33.077240    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode724db28_b13f_4bfa_89ba_98eb630e4241.slice/crio-8264aa0247ae7f6cbd5ab64e6bcb7311b4decb4154d8b89f2f30e7e0361f2a0e WatchSource:0}: Error finding container 8264aa0247ae7f6cbd5ab64e6bcb7311b4decb4154d8b89f2f30e7e0361f2a0e: Status 404 returned error can't find the container with id 8264aa0247ae7f6cbd5ab64e6bcb7311b4decb4154d8b89f2f30e7e0361f2a0e
Oct 30 15:37:33.115832 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.115801    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:37:33.126660 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:33.126637    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1938ddf_e712_417e_9741_8723017995b5.slice/crio-beaf7b8db55c96a489424f1c922d6f37956faf14b0eb79c585d47aca26ba8017 WatchSource:0}: Error finding container beaf7b8db55c96a489424f1c922d6f37956faf14b0eb79c585d47aca26ba8017: Status 404 returned error can't find the container with id beaf7b8db55c96a489424f1c922d6f37956faf14b0eb79c585d47aca26ba8017
Oct 30 15:37:33.131257 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.130215    2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:37:33.141410 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.141390    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:33.144316 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.144294    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:33.151387 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.151360    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lw4vm"
Oct 30 15:37:33.151687 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:33.151666    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89518dc7_2a05_4db9_b4f4_0662fd34d01b.slice/crio-b7a376aef574bec6d3dfe258c9c25ba012dd05d31e973a78c2200b0583131ab5 WatchSource:0}: Error finding container b7a376aef574bec6d3dfe258c9c25ba012dd05d31e973a78c2200b0583131ab5: Status 404 returned error can't find the container with id b7a376aef574bec6d3dfe258c9c25ba012dd05d31e973a78c2200b0583131ab5
Oct 30 15:37:33.155397 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:33.155376    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod587ce512_7067_42a2_9cf1_2babd4feae72.slice/crio-803266f2222e88e04f82849afd6f7142cd2ce34d501ae4454c05a51fb740eb52 WatchSource:0}: Error finding container 803266f2222e88e04f82849afd6f7142cd2ce34d501ae4454c05a51fb740eb52: Status 404 returned error can't find the container with id 803266f2222e88e04f82849afd6f7142cd2ce34d501ae4454c05a51fb740eb52
Oct 30 15:37:33.163439 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:33.163398    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d430d20_3538_482d_8588_a073be77dbd5.slice/crio-0f4bd13535d54ddaa5881bfebf22d7961425d6759fb312230057ec08e165bbc1 WatchSource:0}: Error finding container 0f4bd13535d54ddaa5881bfebf22d7961425d6759fb312230057ec08e165bbc1: Status 404 returned error can't find the container with id 0f4bd13535d54ddaa5881bfebf22d7961425d6759fb312230057ec08e165bbc1
Oct 30 15:37:33.171872 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.171853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:37:33.182222 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:33.182197    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod630d1cfd_5d0a_41ec_bc4d_0d3d6b58b9f8.slice/crio-65cf929f79d064f6edb5c94df150a1e80fb57287fb4f0cc05e0ce7a09f2bb201 WatchSource:0}: Error finding container 65cf929f79d064f6edb5c94df150a1e80fb57287fb4f0cc05e0ce7a09f2bb201: Status 404 returned error can't find the container with id 65cf929f79d064f6edb5c94df150a1e80fb57287fb4f0cc05e0ce7a09f2bb201
Oct 30 15:37:33.182821 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.182798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:37:33.191235 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:37:33.191213    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc7d376_7849_4d50_8c0f_3128e474b54c.slice/crio-94f3402f60b856d21c434e87a91fea567bc6a586c8c8599158c4ae596e5484db WatchSource:0}: Error finding container 94f3402f60b856d21c434e87a91fea567bc6a586c8c8599158c4ae596e5484db: Status 404 returned error can't find the container with id 94f3402f60b856d21c434e87a91fea567bc6a586c8c8599158c4ae596e5484db
Oct 30 15:37:33.193229 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.193209    2577 kuberuntime_manager.go:1707] "Unhandled Error" err="container dns-node-resolver start failed in pod node-resolver-ngvd2_openshift-dns(4cc7d376-7849-4d50-8c0f-3128e474b54c): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Oct 30 15:37:33.193284 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.193243    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-dns/node-resolver-ngvd2" podUID="4cc7d376-7849-4d50-8c0f-3128e474b54c"
Oct 30 15:37:33.340826 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.340784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:33.341000 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.340935    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:33.341000 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.340997    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret podName:75385577-f877-463f-a0fc-c9e2b844e840 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:34.340983062 +0000 UTC m=+3.075341420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret") pod "global-pull-secret-syncer-9l9gb" (UID: "75385577-f877-463f-a0fc-c9e2b844e840") : object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:33.442148 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.441898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:33.442148 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.442109    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:33.442319 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.442175    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs podName:eea4d604-1949-4ba4-964d-e57f74ce431a nodeName:}" failed. No retries permitted until 2026-10-30 15:37:34.442156265 +0000 UTC m=+3.176514614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs") pod "network-metrics-daemon-fqdrs" (UID: "eea4d604-1949-4ba4-964d-e57f74ce431a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:33.542621 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.542582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:33.542817 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.542799    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 15:37:33.542895 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.542825    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 15:37:33.542895 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.542840    2577 projected.go:196] Error preparing data for projected volume kube-api-access-njltq for pod openshift-network-diagnostics/network-check-target-ttqnr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:33.543003 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.542903    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq podName:746ec817-962d-4cb7-9767-0be957f2c533 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:34.542883773 +0000 UTC m=+3.277242116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-njltq" (UniqueName: "kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq") pod "network-check-target-ttqnr" (UID: "746ec817-962d-4cb7-9767-0be957f2c533") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:33.749343 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.749252    2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:37:33.954009 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.953475    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:33.954009 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.953495    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:33.954009 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.953616    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:33.954009 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:33.953740    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:33.975219 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.975138    2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:37:33.988743 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:33.988627    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xrrf9" event={"ID":"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8","Type":"ContainerStarted","Data":"65cf929f79d064f6edb5c94df150a1e80fb57287fb4f0cc05e0ce7a09f2bb201"}
Oct 30 15:37:34.005411 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.005289    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lw4vm" event={"ID":"2d430d20-3538-482d-8588-a073be77dbd5","Type":"ContainerStarted","Data":"0f4bd13535d54ddaa5881bfebf22d7961425d6759fb312230057ec08e165bbc1"}
Oct 30 15:37:34.010761 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.010716    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 15:32:33 +0000 UTC" deadline="2028-06-25 07:32:02.037547251 +0000 UTC"
Oct 30 15:37:34.010761 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.010762    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14487h54m28.026790025s"
Oct 30 15:37:34.028711 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.028674    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" event={"ID":"89518dc7-2a05-4db9-b4f4-0662fd34d01b","Type":"ContainerStarted","Data":"b7a376aef574bec6d3dfe258c9c25ba012dd05d31e973a78c2200b0583131ab5"}
Oct 30 15:37:34.038093 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.038061    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zxvbz" event={"ID":"e724db28-b13f-4bfa-89ba-98eb630e4241","Type":"ContainerStarted","Data":"8264aa0247ae7f6cbd5ab64e6bcb7311b4decb4154d8b89f2f30e7e0361f2a0e"}
Oct 30 15:37:34.053602 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.053535    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4wfrg" event={"ID":"e894be05-f132-4b29-a974-2ddf29c3e991","Type":"ContainerStarted","Data":"5256d5c98bcb08f9b3067d3260527b694c411ece063e65e4fd94954336887d01"}
Oct 30 15:37:34.058271 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.058244    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj" event={"ID":"74a78648-f7a8-4363-9524-8bbc4b6a14a3","Type":"ContainerStarted","Data":"162edece99bb29c7a00db9fdadf2c50e8bdbc11b52679c503295942edb970da0"}
Oct 30 15:37:34.066189 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.066152    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal" event={"ID":"50080b7deadb8e30dee000b7f0bbadfa","Type":"ContainerStarted","Data":"e416e74a75496c398348f99d57a4bf534ea1e1a0a8672a916e4ac8ade6b40f2e"}
Oct 30 15:37:34.082526 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.082485    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ngvd2" event={"ID":"4cc7d376-7849-4d50-8c0f-3128e474b54c","Type":"ContainerStarted","Data":"94f3402f60b856d21c434e87a91fea567bc6a586c8c8599158c4ae596e5484db"}
Oct 30 15:37:34.084366 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:34.084321    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2a8b58f028a9a4a0345ad1f809e47fb9913d7f77c731bfd785819e1c71be9472\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-dns/node-resolver-ngvd2" podUID="4cc7d376-7849-4d50-8c0f-3128e474b54c"
Oct 30 15:37:34.090751 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.090719    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-trlcx" event={"ID":"587ce512-7067-42a2-9cf1-2babd4feae72","Type":"ContainerStarted","Data":"803266f2222e88e04f82849afd6f7142cd2ce34d501ae4454c05a51fb740eb52"}
Oct 30 15:37:34.100641 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.100613    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xhq8" event={"ID":"c1938ddf-e712-417e-9741-8723017995b5","Type":"ContainerStarted","Data":"beaf7b8db55c96a489424f1c922d6f37956faf14b0eb79c585d47aca26ba8017"}
Oct 30 15:37:34.125485 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.125450    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal" event={"ID":"3cd74d1416e17adb1d88ce1b2d585255","Type":"ContainerStarted","Data":"bf1499191d4923a8cb698ff7331daa7d998e88477d80296871135bdfbb9a65c3"}
Oct 30 15:37:34.351195 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.351095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:34.351398 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:34.351272    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:34.351398 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:34.351373    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret podName:75385577-f877-463f-a0fc-c9e2b844e840 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:36.351339373 +0000 UTC m=+5.085697731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret") pod "global-pull-secret-syncer-9l9gb" (UID: "75385577-f877-463f-a0fc-c9e2b844e840") : object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:34.453242 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.453194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:34.453418 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:34.453391    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:34.453525 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:34.453477    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs podName:eea4d604-1949-4ba4-964d-e57f74ce431a nodeName:}" failed. No retries permitted until 2026-10-30 15:37:36.453455025 +0000 UTC m=+5.187813370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs") pod "network-metrics-daemon-fqdrs" (UID: "eea4d604-1949-4ba4-964d-e57f74ce431a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:34.554643 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.554079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:34.554643 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:34.554209    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 15:37:34.554643 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:34.554222    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 15:37:34.554643 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:34.554232    2577 projected.go:196] Error preparing data for projected volume kube-api-access-njltq for pod openshift-network-diagnostics/network-check-target-ttqnr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:34.554643 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:34.554277    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq podName:746ec817-962d-4cb7-9767-0be957f2c533 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:36.554263972 +0000 UTC m=+5.288622319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-njltq" (UniqueName: "kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq") pod "network-check-target-ttqnr" (UID: "746ec817-962d-4cb7-9767-0be957f2c533") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:34.681702 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.681624    2577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:37:34.953764 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:34.953676    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:34.954456 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:34.953843    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:35.012577 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:35.012502    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 15:32:33 +0000 UTC" deadline="2028-06-16 12:15:35.314008864 +0000 UTC"
Oct 30 15:37:35.012577 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:35.012573    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14276h38m0.301439981s"
Oct 30 15:37:35.954822 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:35.954545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:35.954822 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:35.954680    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:35.956831 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:35.956104    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:35.956831 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:35.956222    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:36.369418 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:36.369333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:36.369598 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:36.369575    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:36.369677 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:36.369639    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret podName:75385577-f877-463f-a0fc-c9e2b844e840 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:40.369620251 +0000 UTC m=+9.103978598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret") pod "global-pull-secret-syncer-9l9gb" (UID: "75385577-f877-463f-a0fc-c9e2b844e840") : object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:36.470522 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:36.470486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:36.470672 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:36.470655    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:36.470747 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:36.470727    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs podName:eea4d604-1949-4ba4-964d-e57f74ce431a nodeName:}" failed. No retries permitted until 2026-10-30 15:37:40.47070676 +0000 UTC m=+9.205065114 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs") pod "network-metrics-daemon-fqdrs" (UID: "eea4d604-1949-4ba4-964d-e57f74ce431a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:36.570861 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:36.570824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:36.571039 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:36.571022    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 15:37:36.571104 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:36.571044    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 15:37:36.571104 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:36.571060    2577 projected.go:196] Error preparing data for projected volume kube-api-access-njltq for pod openshift-network-diagnostics/network-check-target-ttqnr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:36.571199 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:36.571122    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq podName:746ec817-962d-4cb7-9767-0be957f2c533 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:40.571100581 +0000 UTC m=+9.305458947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-njltq" (UniqueName: "kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq") pod "network-check-target-ttqnr" (UID: "746ec817-962d-4cb7-9767-0be957f2c533") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:36.954167 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:36.953893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:36.954336 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:36.954302    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:37.163545 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:37.163504    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="1ece63077274e5c1d8c098df81ccd9a4f2d973d47dd45be5c26363fa45542672" exitCode=0
Oct 30 15:37:37.164026 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:37.163561    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xhq8" event={"ID":"c1938ddf-e712-417e-9741-8723017995b5","Type":"ContainerDied","Data":"1ece63077274e5c1d8c098df81ccd9a4f2d973d47dd45be5c26363fa45542672"}
Oct 30 15:37:37.956343 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:37.956260    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:37.956525 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:37.956383    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:37.956818 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:37.956798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:37.956916 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:37.956883    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:38.172064 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:38.172021    2577 generic.go:356] "Generic (PLEG): container finished" podID="3cd74d1416e17adb1d88ce1b2d585255" containerID="f3b76182b402252f9326aedc0d88cadbf9721d8760d64c3fd0732c4f8036671b" exitCode=0
Oct 30 15:37:38.172606 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:38.172134    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal" event={"ID":"3cd74d1416e17adb1d88ce1b2d585255","Type":"ContainerDied","Data":"f3b76182b402252f9326aedc0d88cadbf9721d8760d64c3fd0732c4f8036671b"}
Oct 30 15:37:38.181874 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:38.181842    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal" event={"ID":"50080b7deadb8e30dee000b7f0bbadfa","Type":"ContainerStarted","Data":"d165bcdee810ef3fdc18bb2ac44630fc8301beba40100cdc220623039309f356"}
Oct 30 15:37:38.184956 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:38.184496    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-trlcx" event={"ID":"587ce512-7067-42a2-9cf1-2babd4feae72","Type":"ContainerStarted","Data":"12d3e11e773491130e0a48b656c893a322ce8f75cdef5b72f51baaf0e7223ee2"}
Oct 30 15:37:38.263639 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:38.263568    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal" podStartSLOduration=5.263537501 podStartE2EDuration="5.263537501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:38.26234307 +0000 UTC m=+6.996701437" watchObservedRunningTime="2026-10-30 15:37:38.263537501 +0000 UTC m=+6.997895868"
Oct 30 15:37:38.547775 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:38.547525    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:38.548443 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:38.548202    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:38.592059 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:38.590880    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-trlcx" podStartSLOduration=1.302343088 podStartE2EDuration="5.590860827s" totalImagesPullingTime="4.288517739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:38.304909587 +0000 UTC m=+7.039267953" watchObservedRunningTime="2026-10-30 15:37:38.590860827 +0000 UTC m=+7.325219194"
Oct 30 15:37:38.953502 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:38.953448    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:38.953653 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:38.953593    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:39.193916 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:39.193884    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal" event={"ID":"3cd74d1416e17adb1d88ce1b2d585255","Type":"ContainerStarted","Data":"64ac7c050efae956be088dd8ae6d0b213fd1de509b714a08e720ac7301a703bf"}
Oct 30 15:37:39.197703 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:39.197648    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xrrf9" event={"ID":"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8","Type":"ContainerStarted","Data":"977fe8a32f027d96b96abc9e141bb2fc3e72f63f472508ee3e77ea02488855e0"}
Oct 30 15:37:39.279811 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:39.279284    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal" podStartSLOduration=6.279234541 podStartE2EDuration="6.279234541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:39.238000813 +0000 UTC m=+7.972359178" watchObservedRunningTime="2026-10-30 15:37:39.279234541 +0000 UTC m=+8.013592906"
Oct 30 15:37:39.956718 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:39.955955    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:39.956718 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:39.956100    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:39.956718 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:39.956551    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:39.956718 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:39.956655    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:40.203980 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:40.203944    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj" event={"ID":"74a78648-f7a8-4363-9524-8bbc4b6a14a3","Type":"ContainerStarted","Data":"067913231c43b588aa0116521ef0042ecae6bb8647c0d57ef8ee8e540bee9b5d"}
Oct 30 15:37:40.204531 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:40.204492    2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 15:37:40.403941 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:40.403603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:40.403941 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:40.403762    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:40.403941 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:40.403834    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret podName:75385577-f877-463f-a0fc-c9e2b844e840 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:48.403814071 +0000 UTC m=+17.138172418 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret") pod "global-pull-secret-syncer-9l9gb" (UID: "75385577-f877-463f-a0fc-c9e2b844e840") : object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:40.504914 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:40.504261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:40.504914 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:40.504490    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:40.504914 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:40.504555    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs podName:eea4d604-1949-4ba4-964d-e57f74ce431a nodeName:}" failed. No retries permitted until 2026-10-30 15:37:48.504534977 +0000 UTC m=+17.238893330 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs") pod "network-metrics-daemon-fqdrs" (UID: "eea4d604-1949-4ba4-964d-e57f74ce431a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:40.605020 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:40.604975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:40.605214 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:40.605182    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 15:37:40.605214 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:40.605206    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 15:37:40.605334 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:40.605220    2577 projected.go:196] Error preparing data for projected volume kube-api-access-njltq for pod openshift-network-diagnostics/network-check-target-ttqnr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:40.605334 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:40.605290    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq podName:746ec817-962d-4cb7-9767-0be957f2c533 nodeName:}" failed. No retries permitted until 2026-10-30 15:37:48.605269323 +0000 UTC m=+17.339627668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-njltq" (UniqueName: "kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq") pod "network-check-target-ttqnr" (UID: "746ec817-962d-4cb7-9767-0be957f2c533") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:40.704085 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:40.704055    2577 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 15:37:40.891956 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:40.891794    2577 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T15:37:40.704081753Z","UUID":"d206b4b8-b425-4bfa-a644-32317b1e12a9","Handler":null,"Name":"","Endpoint":""}
Oct 30 15:37:40.893891 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:40.893851    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 15:37:40.893891 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:40.893891    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 15:37:40.953191 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:40.953154    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:40.953497 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:40.953302    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:41.210040 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:41.209996    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj" event={"ID":"74a78648-f7a8-4363-9524-8bbc4b6a14a3","Type":"ContainerStarted","Data":"41c40dce194ac36d5e74d10cf84c5b9c35ed621781173d9bdfccb10e8be1383f"}
Oct 30 15:37:41.954172 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:41.954132    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:41.954387 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:41.954368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:41.954455 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:41.954399    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:41.954533 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:41.954505    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:42.214598 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.214557    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lw4vm" event={"ID":"2d430d20-3538-482d-8588-a073be77dbd5","Type":"ContainerStarted","Data":"271da7251de083d58d9d20f8ea750154c767e9f240a05853acc30d179fba6f74"}
Oct 30 15:37:42.217612 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.217543    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj" event={"ID":"74a78648-f7a8-4363-9524-8bbc4b6a14a3","Type":"ContainerStarted","Data":"dbfd0334c3b25e6596c65dcef879b04f281118b1d14267216fdfae2c919a98b3"}
Oct 30 15:37:42.219845 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.219813    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="f094a52a17f9dffdb2fbc2dcc7bc6b759caff09d0821cc69df009e6e98a92ea6" exitCode=0
Oct 30 15:37:42.219966 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.219855    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xhq8" event={"ID":"c1938ddf-e712-417e-9741-8723017995b5","Type":"ContainerDied","Data":"f094a52a17f9dffdb2fbc2dcc7bc6b759caff09d0821cc69df009e6e98a92ea6"}
Oct 30 15:37:42.260020 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.259954    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-lw4vm" podStartSLOduration=1.1261720689999999 podStartE2EDuration="9.259932972s" totalImagesPullingTime="8.133760903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:42.259305981 +0000 UTC m=+10.993664348" watchObservedRunningTime="2026-10-30 15:37:42.259932972 +0000 UTC m=+10.994291377"
Oct 30 15:37:42.260194 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.260167    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xrrf9" podStartSLOduration=3.6935880660000002 podStartE2EDuration="9.260160177s" totalImagesPullingTime="5.566572111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:39.279611 +0000 UTC m=+8.013969367" watchObservedRunningTime="2026-10-30 15:37:42.260160177 +0000 UTC m=+10.994518543"
Oct 30 15:37:42.355771 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.355664    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj" podStartSLOduration=2.062543371 podStartE2EDuration="10.355648058s" totalImagesPullingTime="8.293104687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:42.355141301 +0000 UTC m=+11.089499668" watchObservedRunningTime="2026-10-30 15:37:42.355648058 +0000 UTC m=+11.090006424"
Oct 30 15:37:42.954068 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.953797    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:42.954293 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:42.954251    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:42.979140 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.979112    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:42.979261 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.979249    2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 15:37:42.980215 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:42.980198    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:37:43.224963 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:43.224491    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zxvbz" event={"ID":"e724db28-b13f-4bfa-89ba-98eb630e4241","Type":"ContainerStarted","Data":"3ec2858a596fabf5aeef90c87a08163c8743daac47803da921b6401ccdac5d99"}
Oct 30 15:37:43.227577 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:43.227451    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="7ddf5be25a25cbd1f40899cbf264f8a2a53ec57c173218de42c2f929874b8659" exitCode=0
Oct 30 15:37:43.227577 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:43.227494    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xhq8" event={"ID":"c1938ddf-e712-417e-9741-8723017995b5","Type":"ContainerDied","Data":"7ddf5be25a25cbd1f40899cbf264f8a2a53ec57c173218de42c2f929874b8659"}
Oct 30 15:37:43.266571 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:43.266523    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zxvbz" podStartSLOduration=2.146573941 podStartE2EDuration="10.266506595s" totalImagesPullingTime="8.119932654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:43.265502039 +0000 UTC m=+11.999860418" watchObservedRunningTime="2026-10-30 15:37:43.266506595 +0000 UTC m=+12.000864959"
Oct 30 15:37:43.953552 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:43.953518    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:43.953713 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:43.953681    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:43.953808 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:43.953713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:43.953863 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:43.953834    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:44.232074 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:44.231969    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4wfrg" event={"ID":"e894be05-f132-4b29-a974-2ddf29c3e991","Type":"ContainerStarted","Data":"e2657109ed285147b997d43c8ea2ac633e2a8e020e10300dd758dc63a5f47ddb"}
Oct 30 15:37:44.234923 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:44.234887    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="e8ce4f234049c8bdc891ea9142474842dea90017e71b9941e71358720cbe031a" exitCode=0
Oct 30 15:37:44.235049 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:44.234959    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xhq8" event={"ID":"c1938ddf-e712-417e-9741-8723017995b5","Type":"ContainerDied","Data":"e8ce4f234049c8bdc891ea9142474842dea90017e71b9941e71358720cbe031a"}
Oct 30 15:37:44.326026 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:44.325971    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4wfrg" podStartSLOduration=2.173749859 podStartE2EDuration="12.325950857s" totalImagesPullingTime="10.152200998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:44.276576 +0000 UTC m=+13.010934364" watchObservedRunningTime="2026-10-30 15:37:44.325950857 +0000 UTC m=+13.060309224"
Oct 30 15:37:44.953262 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:44.953229    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:44.953455 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:44.953396    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:45.953877 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:45.953841    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:45.954608 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:45.954016    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:45.954608 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:45.954070    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:45.954608 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:45.954187    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:46.953026 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:46.952991    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:46.953183 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:46.953128    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:47.249059 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:47.248972    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="9f6875d39162d938468b01f6b96a5aed68716240a1cff29385dd52968d715697" exitCode=0
Oct 30 15:37:47.249059 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:47.249033    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xhq8" event={"ID":"c1938ddf-e712-417e-9741-8723017995b5","Type":"ContainerDied","Data":"9f6875d39162d938468b01f6b96a5aed68716240a1cff29385dd52968d715697"}
Oct 30 15:37:47.953334 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:47.953146    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:47.953441 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:47.953185    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:47.953472 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:47.953451    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:47.953550 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:47.953532    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:48.254966 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.254799    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:37:48.255445 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.255331    2577 generic.go:356] "Generic (PLEG): container finished" podID="89518dc7-2a05-4db9-b4f4-0662fd34d01b" containerID="d3116e1f696328814f57df48db38e126c80066a4d1433013dc3e00a2c25440eb" exitCode=1
Oct 30 15:37:48.255445 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.255389    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" event={"ID":"89518dc7-2a05-4db9-b4f4-0662fd34d01b","Type":"ContainerStarted","Data":"fcb639a9f1d09744843f54fd892fe2439f11957d94bd54a6c27168c79b8fa82d"}
Oct 30 15:37:48.255713 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.255486    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" event={"ID":"89518dc7-2a05-4db9-b4f4-0662fd34d01b","Type":"ContainerStarted","Data":"d232e5a061e62be6c211e35eb3d63f57c7c9aaa8107efda469be02f722bbf82b"}
Oct 30 15:37:48.255713 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.255538    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" event={"ID":"89518dc7-2a05-4db9-b4f4-0662fd34d01b","Type":"ContainerStarted","Data":"164f82496696028d53e2f4f97299f89c89d679101889912f4e8eb6f0fe04e6d8"}
Oct 30 15:37:48.255713 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.255585    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" event={"ID":"89518dc7-2a05-4db9-b4f4-0662fd34d01b","Type":"ContainerStarted","Data":"ab00f386dfbc7d4ab1695ee5d8e4069ecc64c872389519c3ee8b53dff563f244"}
Oct 30 15:37:48.255713 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.255632    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" event={"ID":"89518dc7-2a05-4db9-b4f4-0662fd34d01b","Type":"ContainerDied","Data":"d3116e1f696328814f57df48db38e126c80066a4d1433013dc3e00a2c25440eb"}
Oct 30 15:37:48.255713 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.255677    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" event={"ID":"89518dc7-2a05-4db9-b4f4-0662fd34d01b","Type":"ContainerStarted","Data":"5a7b364678f5477c94da3abd86400bf656a4442b766c9262680f9f47697f70b8"}
Oct 30 15:37:48.259099 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.259073    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="6193e207ac16eaacfb3a5bc26e29612cc90f02f9ac114e97f96a49638de2718e" exitCode=0
Oct 30 15:37:48.259196 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.259111    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xhq8" event={"ID":"c1938ddf-e712-417e-9741-8723017995b5","Type":"ContainerDied","Data":"6193e207ac16eaacfb3a5bc26e29612cc90f02f9ac114e97f96a49638de2718e"}
Oct 30 15:37:48.463976 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.463939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:48.464210 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:48.464109    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:48.464210 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:48.464186    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret podName:75385577-f877-463f-a0fc-c9e2b844e840 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:04.464165443 +0000 UTC m=+33.198523783 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret") pod "global-pull-secret-syncer-9l9gb" (UID: "75385577-f877-463f-a0fc-c9e2b844e840") : object "kube-system"/"original-pull-secret" not registered
Oct 30 15:37:48.565212 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.565120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:48.565370 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:48.565264    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:48.565370 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:48.565330    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs podName:eea4d604-1949-4ba4-964d-e57f74ce431a nodeName:}" failed. No retries permitted until 2026-10-30 15:38:04.565312891 +0000 UTC m=+33.299671244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs") pod "network-metrics-daemon-fqdrs" (UID: "eea4d604-1949-4ba4-964d-e57f74ce431a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 15:37:48.666094 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.666058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:48.666244 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:48.666210    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 15:37:48.666244 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:48.666225    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 15:37:48.666244 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:48.666235    2577 projected.go:196] Error preparing data for projected volume kube-api-access-njltq for pod openshift-network-diagnostics/network-check-target-ttqnr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:48.666331 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:48.666283    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq podName:746ec817-962d-4cb7-9767-0be957f2c533 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:04.666268039 +0000 UTC m=+33.400626395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-njltq" (UniqueName: "kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq") pod "network-check-target-ttqnr" (UID: "746ec817-962d-4cb7-9767-0be957f2c533") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 15:37:48.953850 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:48.953768    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:48.954006 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:48.953877    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:49.262377 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:49.262278    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ngvd2" event={"ID":"4cc7d376-7849-4d50-8c0f-3128e474b54c","Type":"ContainerStarted","Data":"bdf98688ce2f20797a656d59266392af5e367f5b93cb561eafcfd75672dab255"}
Oct 30 15:37:49.265337 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:49.265313    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xhq8" event={"ID":"c1938ddf-e712-417e-9741-8723017995b5","Type":"ContainerStarted","Data":"bb0fa7b2e31c74581f616643116c0c02e4d48d3a2c90747bfda90efcb8a3940f"}
Oct 30 15:37:49.300185 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:49.300140    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-ngvd2" podStartSLOduration=0.537386677 podStartE2EDuration="16.300125255s" totalImagesPullingTime="15.762738578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:49.299946155 +0000 UTC m=+18.034304516" watchObservedRunningTime="2026-10-30 15:37:49.300125255 +0000 UTC m=+18.034483617"
Oct 30 15:37:49.953575 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:49.953546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:49.953735 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:49.953582    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:49.953735 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:49.953655    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:49.953804 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:49.953784    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:50.953146 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:50.953111    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:50.953685 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:50.953228    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:51.271975 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:51.271903    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:37:51.272247 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:51.272227    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" event={"ID":"89518dc7-2a05-4db9-b4f4-0662fd34d01b","Type":"ContainerStarted","Data":"94a4e93b9ef0fced9b75fb4920d35542f2e57ba4eb3f455b4e2407df47b0297b"}
Oct 30 15:37:51.953815 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:51.953781    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:51.954231 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:51.953871    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:51.954231 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:51.953966    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:51.954231 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:51.954053    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:52.953457 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:52.953421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:52.953606 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:52.953535    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:53.281267 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:53.281190    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:37:53.281874 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:53.281538    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" event={"ID":"89518dc7-2a05-4db9-b4f4-0662fd34d01b","Type":"ContainerStarted","Data":"52e103c30261ea7e92fea80d5daaab24dda0d0809fc83909ccee2159d657dc47"}
Oct 30 15:37:53.282014 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:53.281869    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:53.282014 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:53.281899    2577 scope.go:122] "RemoveContainer" containerID="d3116e1f696328814f57df48db38e126c80066a4d1433013dc3e00a2c25440eb"
Oct 30 15:37:53.282014 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:53.281951    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:53.298443 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:53.298421    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:53.298929 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:53.298910    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:37:53.372288 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:53.372230    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9xhq8" podStartSLOduration=9.94658373 podStartE2EDuration="20.372213785s" totalImagesPullingTime="10.425630055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:49.345897579 +0000 UTC m=+18.080255952" watchObservedRunningTime="2026-10-30 15:37:53.372213785 +0000 UTC m=+22.106572145"
Oct 30 15:37:53.953881 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:53.953843    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:53.953881 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:53.953872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:53.954107 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:53.953978    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:53.954158 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:53.954102    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:54.288149 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:54.288065    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:37:54.288717 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:54.288483    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" event={"ID":"89518dc7-2a05-4db9-b4f4-0662fd34d01b","Type":"ContainerStarted","Data":"a3b88d889d9f5a6c5a46c6c025c3f40f9b859fa5daf13bd34d5337e896e7faa3"}
Oct 30 15:37:54.288717 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:54.288574    2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 15:37:54.358768 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:54.358716    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4" podStartSLOduration=6.747728875 podStartE2EDuration="21.358700781s" totalImagesPullingTime="14.610971906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:37:54.358579014 +0000 UTC m=+23.092937404" watchObservedRunningTime="2026-10-30 15:37:54.358700781 +0000 UTC m=+23.093059143"
Oct 30 15:37:54.404087 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:54.404060    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9l9gb"]
Oct 30 15:37:54.404249 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:54.404159    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:54.404249 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:54.404236    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:54.406738 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:54.406713    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ttqnr"]
Oct 30 15:37:54.406872 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:54.406799    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:54.406927 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:54.406890    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:54.416733 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:54.416711    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fqdrs"]
Oct 30 15:37:54.416833 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:54.416806    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:54.416902 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:54.416882    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:55.291091 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:55.291012    2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 15:37:55.953955 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:55.953911    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:55.954122 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:55.953924    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:55.954122 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:55.954056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:55.954122 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:55.954050    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:55.954243 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:55.954129    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:55.954243 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:55.954191    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:57.952976 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:57.952939    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:57.952976 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:57.952967    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:57.953519 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:57.952944    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:57.953519 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:57.953071    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:57.953519 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:57.953161    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:57.953519 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:57.953245    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:59.953387 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:59.953131    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:37:59.953387 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:59.953131    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:37:59.953880 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:59.953422    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fqdrs" podUID="eea4d604-1949-4ba4-964d-e57f74ce431a"
Oct 30 15:37:59.953880 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:59.953139    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:37:59.953880 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:59.953497    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9l9gb" podUID="75385577-f877-463f-a0fc-c9e2b844e840"
Oct 30 15:37:59.953880 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:37:59.953570    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ttqnr" podUID="746ec817-962d-4cb7-9767-0be957f2c533"
Oct 30 15:37:59.996207 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:59.996180    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeReady"
Oct 30 15:37:59.996377 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:37:59.996306    2577 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 15:38:00.080785 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.080737    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k42cc"]
Oct 30 15:38:00.085451 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.085427    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zknrn"]
Oct 30 15:38:00.085574 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.085556    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:00.089971 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.089951    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.111918 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.111895    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k42cc"]
Oct 30 15:38:00.120913 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.120881    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zknrn"]
Oct 30 15:38:00.124468 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.124448    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bxct5"]
Oct 30 15:38:00.128965 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.128950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.132760 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.132739    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-rssdr-2wg26"]
Oct 30 15:38:00.135309 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.135291    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.145974 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.145949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/faef07bf-622a-4f45-8e07-3a64933a6ef8-data-volume\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.146092 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.145981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.146092 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.145997    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hswr\" (UniqueName: \"kubernetes.io/projected/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-api-access-2hswr\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.146202 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.146108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-cert\") pod \"ingress-canary-k42cc\" (UID: \"20a5f545-8ad7-45e5-ae97-1abfeb8d1080\") " pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:00.146202 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.146147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/faef07bf-622a-4f45-8e07-3a64933a6ef8-crio-socket\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.146273 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.146238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.146312 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.146282    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmwx5\" (UniqueName: \"kubernetes.io/projected/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-kube-api-access-fmwx5\") pod \"ingress-canary-k42cc\" (UID: \"20a5f545-8ad7-45e5-ae97-1abfeb8d1080\") " pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:00.152935 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.152913    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bxct5"]
Oct 30 15:38:00.155198 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.155168    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 15:38:00.155304 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.155209    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 15:38:00.155304 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.155259    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 15:38:00.155304 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.155265    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 15:38:00.155436 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.155336    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 15:38:00.156479 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.156457    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 15:38:00.156598 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.156510    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 15:38:00.177625 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.177603    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 15:38:00.179474 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.179450    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 15:38:00.179474 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.179466    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 15:38:00.179474 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.179464    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9c74v\""
Oct 30 15:38:00.179651 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.179522    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 15:38:00.179651 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.179532    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mhkxg\""
Oct 30 15:38:00.179651 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.179522    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rf9nq\""
Oct 30 15:38:00.179651 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.179568    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zt274\""
Oct 30 15:38:00.180410 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.180396    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-rssdr-2wg26"]
Oct 30 15:38:00.247148 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.247148 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-service-ca-bundle\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.247332 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-trusted-ca-bundle\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.247332 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.247332 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a9a6ff0-5946-4635-9f97-c7be77c197c5-tmp-dir\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.247332 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.247313    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 15:38:00.247524 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.247407    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls podName:faef07bf-622a-4f45-8e07-3a64933a6ef8 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:00.7473904 +0000 UTC m=+29.481748753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zknrn" (UID: "faef07bf-622a-4f45-8e07-3a64933a6ef8") : secret "insights-runtime-extractor-tls" not found
Oct 30 15:38:00.247524 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247426    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmwx5\" (UniqueName: \"kubernetes.io/projected/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-kube-api-access-fmwx5\") pod \"ingress-canary-k42cc\" (UID: \"20a5f545-8ad7-45e5-ae97-1abfeb8d1080\") " pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:00.247524 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-archives-path\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.247524 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9a6ff0-5946-4635-9f97-c7be77c197c5-config-volume\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.247704 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/faef07bf-622a-4f45-8e07-3a64933a6ef8-data-volume\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.247704 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.247704 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hswr\" (UniqueName: \"kubernetes.io/projected/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-api-access-2hswr\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.247704 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpqs9\" (UniqueName: \"kubernetes.io/projected/6a9a6ff0-5946-4635-9f97-c7be77c197c5-kube-api-access-fpqs9\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.247704 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.247620    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 15:38:00.247704 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.247686    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm podName:faef07bf-622a-4f45-8e07-3a64933a6ef8 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:00.747669993 +0000 UTC m=+29.482028336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zknrn" (UID: "faef07bf-622a-4f45-8e07-3a64933a6ef8") : configmap "kube-rbac-proxy" not found
Oct 30 15:38:00.247986 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-cert\") pod \"ingress-canary-k42cc\" (UID: \"20a5f545-8ad7-45e5-ae97-1abfeb8d1080\") " pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:00.247986 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/faef07bf-622a-4f45-8e07-3a64933a6ef8-crio-socket\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.247986 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcrnd\" (UniqueName: \"kubernetes.io/projected/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-kube-api-access-mcrnd\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.247986 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247875    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/faef07bf-622a-4f45-8e07-3a64933a6ef8-data-volume\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.247986 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.247940    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/faef07bf-622a-4f45-8e07-3a64933a6ef8-crio-socket\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.251899 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.251879    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-cert\") pod \"ingress-canary-k42cc\" (UID: \"20a5f545-8ad7-45e5-ae97-1abfeb8d1080\") " pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:00.272690 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.272666    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmwx5\" (UniqueName: \"kubernetes.io/projected/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-kube-api-access-fmwx5\") pod \"ingress-canary-k42cc\" (UID: \"20a5f545-8ad7-45e5-ae97-1abfeb8d1080\") " pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:00.272846 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.272826    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hswr\" (UniqueName: \"kubernetes.io/projected/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-api-access-2hswr\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.348329 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.348298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.348329 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.348330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a9a6ff0-5946-4635-9f97-c7be77c197c5-tmp-dir\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.348540 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.348440    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 15:38:00.348540 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.348486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-archives-path\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.348540 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.348497    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls podName:6a9a6ff0-5946-4635-9f97-c7be77c197c5 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:00.848482682 +0000 UTC m=+29.582841026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls") pod "dns-default-bxct5" (UID: "6a9a6ff0-5946-4635-9f97-c7be77c197c5") : secret "dns-default-metrics-tls" not found
Oct 30 15:38:00.348540 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.348522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9a6ff0-5946-4635-9f97-c7be77c197c5-config-volume\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.348734 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.348570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpqs9\" (UniqueName: \"kubernetes.io/projected/6a9a6ff0-5946-4635-9f97-c7be77c197c5-kube-api-access-fpqs9\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.348734 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.348611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcrnd\" (UniqueName: \"kubernetes.io/projected/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-kube-api-access-mcrnd\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.348734 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.348679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-service-ca-bundle\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.348734 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.348682    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a9a6ff0-5946-4635-9f97-c7be77c197c5-tmp-dir\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.348734 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.348703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-trusted-ca-bundle\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.348925 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.348803    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-archives-path\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.349189 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.349172    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-service-ca-bundle\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.349260 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.349237    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9a6ff0-5946-4635-9f97-c7be77c197c5-config-volume\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.349457 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.349442    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-trusted-ca-bundle\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.376993 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.376969    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpqs9\" (UniqueName: \"kubernetes.io/projected/6a9a6ff0-5946-4635-9f97-c7be77c197c5-kube-api-access-fpqs9\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.376993 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.376992    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcrnd\" (UniqueName: \"kubernetes.io/projected/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-kube-api-access-mcrnd\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.399419 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.399395    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:00.446233 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.446204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:00.750873 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.750790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.751037 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.750875    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:00.751037 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.750950    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 15:38:00.751037 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.750964    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 15:38:00.751037 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.751011    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls podName:faef07bf-622a-4f45-8e07-3a64933a6ef8 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:01.750997916 +0000 UTC m=+30.485356256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zknrn" (UID: "faef07bf-622a-4f45-8e07-3a64933a6ef8") : secret "insights-runtime-extractor-tls" not found
Oct 30 15:38:00.751037 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.751025    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm podName:faef07bf-622a-4f45-8e07-3a64933a6ef8 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:01.751019568 +0000 UTC m=+30.485377908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zknrn" (UID: "faef07bf-622a-4f45-8e07-3a64933a6ef8") : configmap "kube-rbac-proxy" not found
Oct 30 15:38:00.758642 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.758619    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k42cc"]
Oct 30 15:38:00.765544 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.763936    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-rssdr-2wg26"]
Oct 30 15:38:00.767246 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:38:00.767225    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a8a0fc_46c9_44ae_b4cf_76e3a17ed1e3.slice/crio-9b9096edeb1ab5811232cdeaaf985cb99938bbbdec270bd2c89a7962a39cc45a WatchSource:0}: Error finding container 9b9096edeb1ab5811232cdeaaf985cb99938bbbdec270bd2c89a7962a39cc45a: Status 404 returned error can't find the container with id 9b9096edeb1ab5811232cdeaaf985cb99938bbbdec270bd2c89a7962a39cc45a
Oct 30 15:38:00.851767 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:00.851731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:00.851914 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.851877    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 15:38:00.851958 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:00.851937    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls podName:6a9a6ff0-5946-4635-9f97-c7be77c197c5 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:01.851922259 +0000 UTC m=+30.586280599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls") pod "dns-default-bxct5" (UID: "6a9a6ff0-5946-4635-9f97-c7be77c197c5") : secret "dns-default-metrics-tls" not found
Oct 30 15:38:01.306051 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.305999    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rssdr-2wg26" event={"ID":"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3","Type":"ContainerStarted","Data":"9b9096edeb1ab5811232cdeaaf985cb99938bbbdec270bd2c89a7962a39cc45a"}
Oct 30 15:38:01.307286 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.307261    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k42cc" event={"ID":"20a5f545-8ad7-45e5-ae97-1abfeb8d1080","Type":"ContainerStarted","Data":"bdc96a647161c0911bf596f6315eb6a35515b5b4d24abf0392512ae47db743cf"}
Oct 30 15:38:01.758916 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.758843    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:01.759056 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.758923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:01.759056 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:01.759025    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 15:38:01.759128 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:01.759075    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm podName:faef07bf-622a-4f45-8e07-3a64933a6ef8 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:03.7590612 +0000 UTC m=+32.493419552 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zknrn" (UID: "faef07bf-622a-4f45-8e07-3a64933a6ef8") : configmap "kube-rbac-proxy" not found
Oct 30 15:38:01.759435 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:01.759415    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 15:38:01.759554 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:01.759471    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls podName:faef07bf-622a-4f45-8e07-3a64933a6ef8 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:03.759456261 +0000 UTC m=+32.493814601 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zknrn" (UID: "faef07bf-622a-4f45-8e07-3a64933a6ef8") : secret "insights-runtime-extractor-tls" not found
Oct 30 15:38:01.859431 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.859392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:01.859602 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:01.859564    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 15:38:01.859672 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:01.859631    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls podName:6a9a6ff0-5946-4635-9f97-c7be77c197c5 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:03.859616929 +0000 UTC m=+32.593975283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls") pod "dns-default-bxct5" (UID: "6a9a6ff0-5946-4635-9f97-c7be77c197c5") : secret "dns-default-metrics-tls" not found
Oct 30 15:38:01.958054 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.958026    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:01.958322 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.958084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:38:01.958411 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.958126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:38:01.975594 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.975570    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 15:38:01.976057 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.975675    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 15:38:01.989013 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.988983    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5ljdx\""
Oct 30 15:38:01.989013 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.989008    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qtl5k\""
Oct 30 15:38:01.989174 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.989081    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 15:38:01.989174 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.989088    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 15:38:01.989174 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:01.989161    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-nlpvl\""
Oct 30 15:38:03.037112 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:03.037077    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:03.037657 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:03.037294    2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 15:38:03.053493 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:03.053470    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:03.776478 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:03.776433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:03.776663 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:03.776530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:03.776727 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:03.776691    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 15:38:03.776788 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:03.776776    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls podName:faef07bf-622a-4f45-8e07-3a64933a6ef8 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:07.776754411 +0000 UTC m=+36.511112765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zknrn" (UID: "faef07bf-622a-4f45-8e07-3a64933a6ef8") : secret "insights-runtime-extractor-tls" not found
Oct 30 15:38:03.776846 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:03.776809    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 15:38:03.776892 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:03.776864    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm podName:faef07bf-622a-4f45-8e07-3a64933a6ef8 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:07.776851451 +0000 UTC m=+36.511209802 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zknrn" (UID: "faef07bf-622a-4f45-8e07-3a64933a6ef8") : configmap "kube-rbac-proxy" not found
Oct 30 15:38:03.877451 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:03.877418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:03.879847 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:03.879819    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:04.042518 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.042434    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:04.316591 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.316504    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rssdr-2wg26" event={"ID":"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3","Type":"ContainerStarted","Data":"ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844"}
Oct 30 15:38:04.318095 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.318072    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k42cc" event={"ID":"20a5f545-8ad7-45e5-ae97-1abfeb8d1080","Type":"ContainerStarted","Data":"bde8d69ff4d09ae6da659c55f569f0229edceb65cd45d4abe837217d38873700"}
Oct 30 15:38:04.356291 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.356235    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-rssdr-2wg26" podStartSLOduration=1.869470628 podStartE2EDuration="4.35622357s" totalImagesPullingTime="2.486752942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:04.355812941 +0000 UTC m=+33.090171306" watchObservedRunningTime="2026-10-30 15:38:04.35622357 +0000 UTC m=+33.090581932"
Oct 30 15:38:04.373294 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.373265    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bxct5"]
Oct 30 15:38:04.383405 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:38:04.383377    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a9a6ff0_5946_4635_9f97_c7be77c197c5.slice/crio-24d34a460f8be378724d6aa245071044c917cb5b8ab8ede677bc2c4e57971a4a WatchSource:0}: Error finding container 24d34a460f8be378724d6aa245071044c917cb5b8ab8ede677bc2c4e57971a4a: Status 404 returned error can't find the container with id 24d34a460f8be378724d6aa245071044c917cb5b8ab8ede677bc2c4e57971a4a
Oct 30 15:38:04.399248 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.399198    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k42cc" podStartSLOduration=1.529807795 podStartE2EDuration="4.39918368s" totalImagesPullingTime="2.869375885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:04.399003722 +0000 UTC m=+33.133362083" watchObservedRunningTime="2026-10-30 15:38:04.39918368 +0000 UTC m=+33.133542042"
Oct 30 15:38:04.482764 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.482730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:04.485048 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.485025    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:04.583524 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.583446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:38:04.585712 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.585694    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:38:04.684586 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.684549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:38:04.687042 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.687013    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:38:04.699704 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.699682    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:04.731823 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.731798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:38:04.746219 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:04.746202    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:38:05.049122 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:05.049091    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9l9gb"]
Oct 30 15:38:05.052863 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:38:05.052829    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75385577_f877_463f_a0fc_c9e2b844e840.slice/crio-048de475f8b290e82c77d99a0d039d251157580efe3b3a4b438f0e8474e01f14 WatchSource:0}: Error finding container 048de475f8b290e82c77d99a0d039d251157580efe3b3a4b438f0e8474e01f14: Status 404 returned error can't find the container with id 048de475f8b290e82c77d99a0d039d251157580efe3b3a4b438f0e8474e01f14
Oct 30 15:38:05.076304 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:05.076255    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fqdrs"]
Oct 30 15:38:05.079745 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:38:05.079718    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea4d604_1949_4ba4_964d_e57f74ce431a.slice/crio-92872077501cceac18b31f6e598a4ee0a7f4ca76db467b55a4d0082cb9a8fff0 WatchSource:0}: Error finding container 92872077501cceac18b31f6e598a4ee0a7f4ca76db467b55a4d0082cb9a8fff0: Status 404 returned error can't find the container with id 92872077501cceac18b31f6e598a4ee0a7f4ca76db467b55a4d0082cb9a8fff0
Oct 30 15:38:05.101676 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:05.101654    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ttqnr"]
Oct 30 15:38:05.103127 ip-10-0-140-110 kubenswrapper[2577]: W1030 15:38:05.103092    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746ec817_962d_4cb7_9767_0be957f2c533.slice/crio-05854b46ac6e3336a8bed1009b92cc45799e3ffe9ad66d2312c3d3ac9d60e213 WatchSource:0}: Error finding container 05854b46ac6e3336a8bed1009b92cc45799e3ffe9ad66d2312c3d3ac9d60e213: Status 404 returned error can't find the container with id 05854b46ac6e3336a8bed1009b92cc45799e3ffe9ad66d2312c3d3ac9d60e213
Oct 30 15:38:05.321661 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:05.321627    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fqdrs" event={"ID":"eea4d604-1949-4ba4-964d-e57f74ce431a","Type":"ContainerStarted","Data":"92872077501cceac18b31f6e598a4ee0a7f4ca76db467b55a4d0082cb9a8fff0"}
Oct 30 15:38:05.322783 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:05.322756    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9l9gb" event={"ID":"75385577-f877-463f-a0fc-c9e2b844e840","Type":"ContainerStarted","Data":"048de475f8b290e82c77d99a0d039d251157580efe3b3a4b438f0e8474e01f14"}
Oct 30 15:38:05.323886 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:05.323866    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bxct5" event={"ID":"6a9a6ff0-5946-4635-9f97-c7be77c197c5","Type":"ContainerStarted","Data":"24d34a460f8be378724d6aa245071044c917cb5b8ab8ede677bc2c4e57971a4a"}
Oct 30 15:38:05.324892 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:05.324872    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ttqnr" event={"ID":"746ec817-962d-4cb7-9767-0be957f2c533","Type":"ContainerStarted","Data":"05854b46ac6e3336a8bed1009b92cc45799e3ffe9ad66d2312c3d3ac9d60e213"}
Oct 30 15:38:06.370964 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:06.370925    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngvd2_4cc7d376-7849-4d50-8c0f-3128e474b54c/dns-node-resolver/0.log"
Oct 30 15:38:07.333158 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.333117    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fqdrs" event={"ID":"eea4d604-1949-4ba4-964d-e57f74ce431a","Type":"ContainerStarted","Data":"2865f236d60c64f3098016e93003ed39f8b0ecdb455c4b0ee177e5690db0c4d8"}
Oct 30 15:38:07.333335 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.333221    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fqdrs" event={"ID":"eea4d604-1949-4ba4-964d-e57f74ce431a","Type":"ContainerStarted","Data":"a338763b6ffc234153ad8d87c414224e8388d45baa9c7fa651ae288134a6f1e2"}
Oct 30 15:38:07.336038 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.336011    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bxct5" event={"ID":"6a9a6ff0-5946-4635-9f97-c7be77c197c5","Type":"ContainerStarted","Data":"14d6253abbda84ef6b285735b530f0152b14daa2cf5c1551dfc14ee5da0cdf05"}
Oct 30 15:38:07.336169 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.336060    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bxct5" event={"ID":"6a9a6ff0-5946-4635-9f97-c7be77c197c5","Type":"ContainerStarted","Data":"c97c9bd6131ca2bc06d2c5d4c3586f90f8c8ed83cb5604ed5cfd5f884e088b92"}
Oct 30 15:38:07.336169 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.336145    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:07.371389 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.371338    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xrrf9_630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8/node-ca/0.log"
Oct 30 15:38:07.375769 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.375703    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fqdrs" podStartSLOduration=32.544734306 podStartE2EDuration="34.375685457s" totalImagesPullingTime="1.830951151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:07.374469863 +0000 UTC m=+36.108828226" watchObservedRunningTime="2026-10-30 15:38:07.375685457 +0000 UTC m=+36.110043820"
Oct 30 15:38:07.418266 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.418167    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-bxct5" podStartSLOduration=4.893438537 podStartE2EDuration="7.418152902s" totalImagesPullingTime="2.524714365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:07.417746105 +0000 UTC m=+36.152104466" watchObservedRunningTime="2026-10-30 15:38:07.418152902 +0000 UTC m=+36.152511264"
Oct 30 15:38:07.809907 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.809829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:07.809907 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.809887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:07.810112 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:07.810000    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 15:38:07.810112 ip-10-0-140-110 kubenswrapper[2577]: E1030 15:38:07.810086    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls podName:faef07bf-622a-4f45-8e07-3a64933a6ef8 nodeName:}" failed. No retries permitted until 2026-10-30 15:38:15.81006858 +0000 UTC m=+44.544426920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zknrn" (UID: "faef07bf-622a-4f45-8e07-3a64933a6ef8") : secret "insights-runtime-extractor-tls" not found
Oct 30 15:38:07.810563 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.810534    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:07.970883 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:07.970832    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k42cc_20a5f545-8ad7-45e5-ae97-1abfeb8d1080/serve-healthcheck-canary/0.log"
Oct 30 15:38:09.343801 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:09.343762    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ttqnr" event={"ID":"746ec817-962d-4cb7-9767-0be957f2c533","Type":"ContainerStarted","Data":"4e7a37ce258c252f064772b0ece9a670ed2169e0b68bde126c221541353f8d23"}
Oct 30 15:38:09.344207 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:09.343912    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:38:09.382892 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:09.382833    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ttqnr" podStartSLOduration=32.646565272 podStartE2EDuration="36.382819442s" totalImagesPullingTime="3.73625417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:09.382128992 +0000 UTC m=+38.116487355" watchObservedRunningTime="2026-10-30 15:38:09.382819442 +0000 UTC m=+38.117177813"
Oct 30 15:38:10.352890 ip-10-0-140-110 kubenswrapper[2577]: I1030 15:38:10.352838    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9l9gb" event={"ID":"75385577-f877-463f-a0fc-c9e2b844e840","Type":"ContainerStarted","Data":"f37a65abdc3196009c72afbe5d4fb8f29307ba69e74f91a02010ec6ddd55dd16"}
Oct 30 15:38:10.512064 ip-10-0-140-110 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 15:38:10.522724 ip-10-0-140-110 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 15:38:10.522886 ip-10-0-140-110 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 15:38:10.522953 ip-10-0-140-110 systemd[1]: kubelet.service: Consumed 2.411s CPU time.
Oct 30 15:38:10.533942 ip-10-0-140-110 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 15:38:10.628729 ip-10-0-140-110 kubenswrapper[4585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 15:38:10.628729 ip-10-0-140-110 kubenswrapper[4585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 15:38:10.628729 ip-10-0-140-110 kubenswrapper[4585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 15:38:10.628729 ip-10-0-140-110 kubenswrapper[4585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 15:38:10.630857 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630760    4585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:38:10.630857 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630848    4585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:38:10.630857 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630851    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:38:10.630857 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630854    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:38:10.630857 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630857    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:38:10.630857 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630860    4585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:38:10.630857 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630867    4585 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630870    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630873    4585 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630875    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630878    4585 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630881    4585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630884    4585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630887    4585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630889    4585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630892    4585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630894    4585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630897    4585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630902    4585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630906    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630910    4585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630913    4585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630916    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630919    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630922    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:38:10.631129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630925    4585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630928    4585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630930    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630933    4585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630936    4585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630938    4585 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630941    4585 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630943    4585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630946    4585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630949    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630951    4585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630954    4585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630956    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630959    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630963    4585 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630966    4585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630968    4585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630971    4585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630974    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630976    4585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630979    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:38:10.631664 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630982    4585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630984    4585 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630987    4585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630990    4585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630992    4585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630995    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.630997    4585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631000    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631002    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631005    4585 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631007    4585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631010    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631013    4585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631016    4585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631018    4585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631021    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631023    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631026    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631029    4585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631031    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:38:10.632194 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631034    4585 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631037    4585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631039    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631043    4585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631047    4585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631050    4585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631054    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631057    4585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631060    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631062    4585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631065    4585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631067    4585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631070    4585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631073    4585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631076    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631078    4585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631081    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631084    4585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631086    4585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:38:10.632700 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631090    4585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631092    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631095    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631097    4585 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631100    4585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631103    4585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631106    4585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631108    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631112    4585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631114    4585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631117    4585 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631120    4585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631123    4585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631125    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631128    4585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631130    4585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631134    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631137    4585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631139    4585 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:38:10.633196 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631142    4585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631146    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631962    4585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631971    4585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631974    4585 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631977    4585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631980    4585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631983    4585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631985    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631988    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631991    4585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631993    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631996    4585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.631999    4585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632001    4585 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632004    4585 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632012    4585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632015    4585 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632018    4585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632020    4585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:38:10.633682 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632023    4585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632026    4585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632028    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632031    4585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632034    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632036    4585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632039    4585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632041    4585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632044    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632046    4585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632049    4585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632052    4585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632054    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632056    4585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632059    4585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632064    4585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632068    4585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632071    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632074    4585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:38:10.634189 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632077    4585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632080    4585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632082    4585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632085    4585 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632087    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632090    4585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632092    4585 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632095    4585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632098    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632102    4585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632104    4585 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632108    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632110    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632113    4585 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632116    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632118    4585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632121    4585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632123    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632128    4585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:38:10.639883 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632132    4585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632135    4585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632138    4585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632140    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632143    4585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632146    4585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632148    4585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632151    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632153    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632157    4585 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632160    4585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632162    4585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632165    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632167    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632170    4585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632173    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632175    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632178    4585 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632180    4585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632183    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:38:10.640396 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632185    4585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632188    4585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632192    4585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632195    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632198    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632201    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632204    4585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632207    4585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632209    4585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632212    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632214    4585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632217    4585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632220    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632222    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632225    4585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632227    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632230    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632232    4585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632235    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:38:10.640906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632237    4585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632240    4585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632242    4585 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632245    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632248    4585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632252    4585 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632255    4585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632257    4585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632260    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632262    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632264    4585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632755    4585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632761    4585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632764    4585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632766    4585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632770    4585 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632773    4585 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632775    4585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632778    4585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632781    4585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:38:10.641381 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632783    4585 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632786    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632789    4585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632791    4585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632794    4585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632796    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632799    4585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632801    4585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632804    4585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632806    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632809    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632811    4585 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632814    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632816    4585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632819    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632822    4585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632824    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632827    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632829    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:38:10.642036 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632832    4585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632835    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632838    4585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632840    4585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632845    4585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632849    4585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632852    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632854    4585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632857    4585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632860    4585 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632862    4585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632865    4585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632868    4585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632870    4585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632873    4585 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632875    4585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632878    4585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632880    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632883    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:38:10.642502 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632885    4585 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632888    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632891    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632893    4585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632896    4585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632898    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632901    4585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632903    4585 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632906    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632909    4585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632912    4585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632915    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632918    4585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632920    4585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632922    4585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632925    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632927    4585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632930    4585 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632932    4585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632935    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:38:10.642972 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632937    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632939    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632943    4585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632945    4585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632948    4585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632951    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632954    4585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632956    4585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632959    4585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632961    4585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632963    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632966    4585 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632969    4585 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632971    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632974    4585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632976    4585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632979    4585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632981    4585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632984    4585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632986    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:38:10.643518 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632989    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632991    4585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.632993    4585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633003    4585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633006    4585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633010    4585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633014    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633017    4585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633020    4585 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633022    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633026    4585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633029    4585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633032    4585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633034    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633037    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633040    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633042    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633045    4585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633047    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633142    4585 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 15:38:10.644228 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633149    4585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633155    4585 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633162    4585 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633171    4585 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633176    4585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633183    4585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633189    4585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633192    4585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633195    4585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633199    4585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633203    4585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633207    4585 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633210    4585 flags.go:64] FLAG: --cgroup-root=""
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633213    4585 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633216    4585 flags.go:64] FLAG: --client-ca-file=""
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633219    4585 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633223    4585 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633228    4585 flags.go:64] FLAG: --cluster-domain=""
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633231    4585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633234    4585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633238    4585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633242    4585 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633246    4585 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 15:38:10.644895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633249    4585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633252    4585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633256    4585 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633259    4585 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633262    4585 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633265    4585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633268    4585 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633271    4585 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633276    4585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633279    4585 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633282    4585 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633285    4585 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633290    4585 flags.go:64] FLAG: --enable-server="true"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633293    4585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633297    4585 flags.go:64] FLAG: --event-burst="100"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633300    4585 flags.go:64] FLAG: --event-qps="50"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633303    4585 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633306    4585 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633310    4585 flags.go:64] FLAG: --eviction-hard=""
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633314    4585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633317    4585 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633320    4585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633323    4585 flags.go:64] FLAG: --eviction-soft=""
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633327    4585 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633330    4585 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 15:38:10.646002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633333    4585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633336    4585 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633339    4585 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633342    4585 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633360    4585 flags.go:64] FLAG: --feature-gates=""
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633365    4585 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633368    4585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633371    4585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633374    4585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633378    4585 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633381    4585 flags.go:64] FLAG: --help="false"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633385    4585 flags.go:64] FLAG: --hostname-override="ip-10-0-140-110.ec2.internal"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633388    4585 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633391    4585 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633394    4585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633398    4585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633401    4585 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633404    4585 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633408    4585 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633412    4585 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633415    4585 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633418    4585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633424    4585 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633427    4585 flags.go:64] FLAG: --kube-reserved=""
Oct 30 15:38:10.646791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633430    4585 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633433    4585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633436    4585 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633439    4585 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633442    4585 flags.go:64] FLAG: --lock-file=""
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633445    4585 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633448    4585 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633451    4585 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633457    4585 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633460    4585 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633463    4585 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633467    4585 flags.go:64] FLAG: --logging-format="text"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633470    4585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633473    4585 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633476    4585 flags.go:64] FLAG: --manifest-url=""
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633479    4585 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633484    4585 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633487    4585 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633491    4585 flags.go:64] FLAG: --max-pods="110"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633494    4585 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633497    4585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633500    4585 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633503    4585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633507    4585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633510    4585 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 15:38:10.647454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633512    4585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633519    4585 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633525    4585 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633528    4585 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633531    4585 flags.go:64] FLAG: --pod-cidr=""
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633535    4585 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633538    4585 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633542    4585 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633545    4585 flags.go:64] FLAG: --port="10250"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633548    4585 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633551    4585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0609ecc1757f97782"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633554    4585 flags.go:64] FLAG: --qos-reserved=""
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633557    4585 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633561    4585 flags.go:64] FLAG: --register-node="true"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633564    4585 flags.go:64] FLAG: --register-with-taints=""
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633568    4585 flags.go:64] FLAG: --registry-burst="10"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633571    4585 flags.go:64] FLAG: --registry-qps="5"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633573    4585 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633577    4585 flags.go:64] FLAG: --reserved-memory=""
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633584    4585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633587    4585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633590    4585 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633593    4585 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633596    4585 flags.go:64] FLAG: --runonce="false"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633600    4585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633603    4585 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633606    4585 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 15:38:10.648074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633609    4585 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633612    4585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633615    4585 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633618    4585 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633622    4585 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633624    4585 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633628    4585 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633632    4585 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633635    4585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633638    4585 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633642    4585 flags.go:64] FLAG: --system-cgroups=""
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633645    4585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633651    4585 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633654    4585 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633657    4585 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633661    4585 flags.go:64] FLAG: --tls-min-version=""
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633664    4585 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633667    4585 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633670    4585 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633673    4585 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633676    4585 flags.go:64] FLAG: --v="2"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633681    4585 flags.go:64] FLAG: --version="false"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633685    4585 flags.go:64] FLAG: --vmodule=""
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633689    4585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.633694    4585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 15:38:10.648754 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633830    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633833    4585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633836    4585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633839    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633841    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633844    4585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633847    4585 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633849    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633852    4585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633855    4585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633857    4585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633860    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633863    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633865    4585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633869    4585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633872    4585 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633874    4585 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633878    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633881    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633884    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:38:10.649371 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633886    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633889    4585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633892    4585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633895    4585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633898    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633901    4585 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633904    4585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633906    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633909    4585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633923    4585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633926    4585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633928    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633932    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633934    4585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633939    4585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633942    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633946    4585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633949    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633951    4585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:38:10.649868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633954    4585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633956    4585 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633959    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633962    4585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633965    4585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633968    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633970    4585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633974    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633977    4585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633979    4585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633983    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633986    4585 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633989    4585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633991    4585 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633994    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.633997    4585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634000    4585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634002    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634005    4585 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634008    4585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:38:10.650334 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634010    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634013    4585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634015    4585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634018    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634020    4585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634023    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634026    4585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634029    4585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634031    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634034    4585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634037    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634039    4585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634042    4585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634045    4585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634047    4585 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634050    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634052    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634055    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634057    4585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:38:10.650836 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634061    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634064    4585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634066    4585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634070    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634073    4585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634075    4585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634079    4585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634083    4585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634086    4585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634089    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634092    4585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634095    4585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634097    4585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634100    4585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634102    4585 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634105    4585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634108    4585 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634110    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634113    4585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:38:10.651304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634116    4585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634122    4585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634125    4585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634128    4585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634130    4585 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634133    4585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634135    4585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634138    4585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634140    4585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.634167    4585 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634206    4585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634211    4585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634214    4585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634217    4585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634221    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:38:10.651796 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634224    4585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634228    4585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634231    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634234    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634236    4585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634239    4585 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634241    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634244    4585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634247    4585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634249    4585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634252    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634255    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634257    4585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634260    4585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634262    4585 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634265    4585 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634268    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634270    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634273    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634276    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:38:10.652179 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634279    4585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634282    4585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634285    4585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634287    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634290    4585 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634293    4585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634295    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634298    4585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634300    4585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634303    4585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634305    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634309    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634312    4585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634317    4585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634322    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634324    4585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634327    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634329    4585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634332    4585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:38:10.652722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634334    4585 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634337    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634340    4585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634355    4585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634358    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634361    4585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634363    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634366    4585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634369    4585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634372    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634374    4585 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634377    4585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634380    4585 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634383    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634386    4585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634388    4585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634391    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634394    4585 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634397    4585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634400    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:38:10.653198 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634402    4585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634405    4585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634407    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634410    4585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634415    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634418    4585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634422    4585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634424    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634427    4585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634430    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634433    4585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634436    4585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634438    4585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634441    4585 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634444    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634446    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634449    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634451    4585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634454    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634457    4585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:38:10.653715 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634459    4585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634462    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634464    4585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634467    4585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634471    4585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634475    4585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634478    4585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634481    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634483    4585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634486    4585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634489    4585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634491    4585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634494    4585 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634496    4585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634499    4585 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634502    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634506    4585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634509    4585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634513    4585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:38:10.654208 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634517    4585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634520    4585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.634522    4585 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.638184    4585 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.638199    4585 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.638217    4585 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.638222    4585 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638281    4585 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638286    4585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638289    4585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638293    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638295    4585 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638298    4585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638301    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638303    4585 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638306    4585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638309    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638311    4585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638314    4585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638372    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:38:10.654701 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638440    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638444    4585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638447    4585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638450    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638454    4585 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638457    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638460    4585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638464    4585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638466    4585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638469    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638472    4585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638475    4585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638478    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638484    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638486    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638489    4585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638492    4585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638495    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638498    4585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638501    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:38:10.655186 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638504    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638507    4585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638509    4585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638512    4585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638515    4585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638518    4585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638523    4585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638526    4585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638533    4585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638536    4585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638539    4585 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638542    4585 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638545    4585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638549    4585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638551    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638555    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638558    4585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638561    4585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638567    4585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638570    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:38:10.655951 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638572    4585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638576    4585 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638578    4585 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638581    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638584    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638587    4585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638590    4585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638593    4585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638596    4585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638599    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638602    4585 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638607    4585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638610    4585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638613    4585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638616    4585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638618    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638621    4585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638624    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638627    4585 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:38:10.656578 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638630    4585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638636    4585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638640    4585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638644    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638650    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638653    4585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638656    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638659    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638662    4585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638665    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638668    4585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638671    4585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638674    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638677    4585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638680    4585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638684    4585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638688    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638695    4585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638698    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:38:10.657066 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638701    4585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638704    4585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638707    4585 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638710    4585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638712    4585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638715    4585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638718    4585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638721    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638724    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638727    4585 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638733    4585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638736    4585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638738    4585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638741    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.638747    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.638798    4585 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 15:38:10.657726 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639109    4585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639119    4585 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639124    4585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639128    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639134    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639138    4585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639143    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639147    4585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639151    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639158    4585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639163    4585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639168    4585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639172    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639182    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639186    4585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639190    4585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639195    4585 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639199    4585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639205    4585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:38:10.658221 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639211    4585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639216    4585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639221    4585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639225    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639229    4585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639234    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639243    4585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639247    4585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639252    4585 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639257    4585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639261    4585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639268    4585 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639272    4585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639276    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639281    4585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639285    4585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639289    4585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639293    4585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639303    4585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:38:10.658708 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639312    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639316    4585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639320    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639327    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639331    4585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639335    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639339    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639359    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639364    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639381    4585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639386    4585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639390    4585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639399    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639404    4585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639408    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639413    4585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639417    4585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639421    4585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639426    4585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:38:10.659170 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639430    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639434    4585 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639437    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639440    4585 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639443    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639447    4585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639472    4585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639479    4585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639482    4585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639485    4585 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639488    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639508    4585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639511    4585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639518    4585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639604    4585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639609    4585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639613    4585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639616    4585 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639619    4585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639621    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:38:10.659689 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639624    4585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639627    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639630    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639633    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639636    4585 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639639    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639641    4585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639644    4585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639647    4585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639649    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639652    4585 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639655    4585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639657    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639660    4585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639662    4585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639665    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639668    4585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639670    4585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639674    4585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639677    4585 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:38:10.660199 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639679    4585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639682    4585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639684    4585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639687    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639690    4585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639692    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639696    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639698    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639703    4585 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.639739    4585 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639781    4585 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639787    4585 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639790    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639793    4585 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639796    4585 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639798    4585 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 15:38:10.660740 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639803    4585 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639806    4585 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639808    4585 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639811    4585 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639814    4585 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639816    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639819    4585 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639823    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639826    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639828    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639831    4585 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639835    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639837    4585 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639840    4585 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639844    4585 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639847    4585 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639849    4585 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639853    4585 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639855    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639858    4585 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 15:38:10.661136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639861    4585 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639863    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639866    4585 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639869    4585 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639871    4585 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639875    4585 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639878    4585 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639880    4585 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639883    4585 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639886    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639888    4585 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639891    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639894    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639897    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639900    4585 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639903    4585 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639905    4585 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639908    4585 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639911    4585 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639913    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 15:38:10.661679 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639916    4585 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639919    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639921    4585 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639924    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639927    4585 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639929    4585 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639933    4585 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639936    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639939    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639941    4585 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639944    4585 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639946    4585 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639949    4585 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639954    4585 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639958    4585 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639961    4585 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639964    4585 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639968    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639971    4585 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 15:38:10.662163 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639974    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639976    4585 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639979    4585 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639982    4585 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639984    4585 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639987    4585 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639989    4585 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639992    4585 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639994    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639997    4585 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.639999    4585 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640002    4585 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640004    4585 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640007    4585 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640010    4585 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640013    4585 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640015    4585 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640018    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640020    4585 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 15:38:10.662640 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640024    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640027    4585 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640029    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640032    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640035    4585 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640038    4585 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640040    4585 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640043    4585 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640045    4585 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640048    4585 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640050    4585 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640055    4585 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640057    4585 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640061    4585 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640064    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640066    4585 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640069    4585 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640072    4585 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640074    4585 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 15:38:10.663088 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640079    4585 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640082    4585 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:10.640085    4585 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.640144    4585 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.641131    4585 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.641199    4585 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.641391    4585 server.go:1014] "Starting client certificate rotation"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.641542    4585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.641607    4585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 15:32:32 +0000 UTC" deadline="2028-07-08 13:20:23.44835459 +0000 UTC"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.641649    4585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14805h42m12.806709477s"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.641871    4585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.642949    4585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.644203    4585 log.go:25] "Validated CRI v1 runtime API"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.646191    4585 log.go:25] "Validated CRI v1 image API"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.646931    4585 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 15:38:10.663651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.648482    4585 fs.go:135] Filesystem UUIDs: map[2df379d2-e4bf-4b2b-8874-576ed76ba068:/dev/nvme0n1p3 388aa26c-e611-43aa-9553-ffac4cc82e87:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 30 15:38:10.663964 ip-10-0-140-110 systemd[1]: Started Kubernetes Kubelet.
Oct 30 15:38:10.664440 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.648501    4585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/048de475f8b290e82c77d99a0d039d251157580efe3b3a4b438f0e8474e01f14/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/048de475f8b290e82c77d99a0d039d251157580efe3b3a4b438f0e8474e01f14/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/05854b46ac6e3336a8bed1009b92cc45799e3ffe9ad66d2312c3d3ac9d60e213/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/05854b46ac6e3336a8bed1009b92cc45799e3ffe9ad66d2312c3d3ac9d60e213/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0f4bd13535d54ddaa5881bfebf22d7961425d6759fb312230057ec08e165bbc1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0f4bd13535d54ddaa5881bfebf22d7961425d6759fb312230057ec08e165bbc1/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/162edece99bb29c7a00db9fdadf2c50e8bdbc11b52679c503295942edb970da0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/162edece99bb29c7a00db9fdadf2c50e8bdbc11b52679c503295942edb970da0/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/24d34a460f8be378724d6aa245071044c917cb5b8ab8ede677bc2c4e57971a4a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/24d34a460f8be378724d6aa245071044c917cb5b8ab8ede677bc2c4e57971a4a/userdata/shm major:0 minor:225 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/65cf929f79d064f6edb5c94df150a1e80fb57287fb4f0cc05e0ce7a09f2bb201/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/65cf929f79d064f6edb5c94df150a1e80fb57287fb4f0cc05e0ce7a09f2bb201/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/803266f2222e88e04f82849afd6f7142cd2ce34d501ae4454c05a51fb740eb52/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/803266f2222e88e04f82849afd6f7142cd2ce34d501ae4454c05a51fb740eb52/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8264aa0247ae7f6cbd5ab64e6bcb7311b4decb4154d8b89f2f30e7e0361f2a0e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8264aa0247ae7f6cbd5ab64e6bcb7311b4decb4154d8b89f2f30e7e0361f2a0e/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/92872077501cceac18b31f6e598a4ee0a7f4ca76db467b55a4d0082cb9a8fff0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/92872077501cceac18b31f6e598a4ee0a7f4ca76db467b55a4d0082cb9a8fff0/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/94f3402f60b856d21c434e87a91fea567bc6a586c8c8599158c4ae596e5484db/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/94f3402f60b856d21c434e87a91fea567bc6a586c8c8599158c4ae596e5484db/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9b9096edeb1ab5811232cdeaaf985cb99938bbbdec270bd2c89a7962a39cc45a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9b9096edeb1ab5811232cdeaaf985cb99938bbbdec270bd2c89a7962a39cc45a/userdata/shm major:0 minor:206 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b7a376aef574bec6d3dfe258c9c25ba012dd05d31e973a78c2200b0583131ab5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b7a376aef574bec6d3dfe258c9c25ba012dd05d31e973a78c2200b0583131ab5/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bdc96a647161c0911bf596f6315eb6a35515b5b4d24abf0392512ae47db743cf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bdc96a647161c0911bf596f6315eb6a35515b5b4d24abf0392512ae47db743cf/userdata/shm major:0 minor:204 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/beaf7b8db55c96a489424f1c922d6f37956faf14b0eb79c585d47aca26ba8017/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/beaf7b8db55c96a489424f1c922d6f37956faf14b0eb79c585d47aca26ba8017/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bf1499191d4923a8cb698ff7331daa7d998e88477d80296871135bdfbb9a65c3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bf1499191d4923a8cb698ff7331daa7d998e88477d80296871135bdfbb9a65c3/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e416e74a75496c398348f99d57a4bf534ea1e1a0a8672a916e4ac8ade6b40f2e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e416e74a75496c398348f99d57a4bf534ea1e1a0a8672a916e4ac8ade6b40f2e/userdata/shm major:0 minor:64 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3/volumes/kubernetes.io~projected/kube-api-access-mcrnd:{mountpoint:/var/lib/kubelet/pods/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3/volumes/kubernetes.io~projected/kube-api-access-mcrnd major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20a5f545-8ad7-45e5-ae97-1abfeb8d1080/volumes/kubernetes.io~projected/kube-api-access-fmwx5:{mountpoint:/var/lib/kubelet/pods/20a5f545-8ad7-45e5-ae97-1abfeb8d1080/volumes/kubernetes.io~projected/kube-api-access-fmwx5 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/20a5f545-8ad7-45e5-ae97-1abfeb8d1080/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/20a5f545-8ad7-45e5-ae97-1abfeb8d1080/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2d430d20-3538-482d-8588-a073be77dbd5/volumes/kubernetes.io~projected/kube-api-access-5qrm4:{mountpoint:/var/lib/kubelet/pods/2d430d20-3538-482d-8588-a073be77dbd5/volumes/kubernetes.io~projected/kube-api-access-5qrm4 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4cc7d376-7849-4d50-8c0f-3128e474b54c/volumes/kubernetes.io~projected/kube-api-access-nrsng:{mountpoint:/var/lib/kubelet/pods/4cc7d376-7849-4d50-8c0f-3128e474b54c/volumes/kubernetes.io~projected/kube-api-access-nrsng major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/587ce512-7067-42a2-9cf1-2babd4feae72/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/587ce512-7067-42a2-9cf1-2babd4feae72/volumes/kubernetes.io~secret/agent-certs major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8/volumes/kubernetes.io~projected/kube-api-access-zmmnk:{mountpoint:/var/lib/kubelet/pods/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8/volumes/kubernetes.io~projected/kube-api-access-zmmnk major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6a9a6ff0-5946-4635-9f97-c7be77c197c5/volumes/kubernetes.io~projected/kube-api-access-fpqs9:{mountpoint:/var/lib/kubelet/pods/6a9a6ff0-5946-4635-9f97-c7be77c197c5/volumes/kubernetes.io~projected/kube-api-access-fpqs9 major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6a9a6ff0-5946-4635-9f97-c7be77c197c5/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/6a9a6ff0-5946-4635-9f97-c7be77c197c5/volumes/kubernetes.io~secret/metrics-tls major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/746ec817-962d-4cb7-9767-0be957f2c533/volumes/kubernetes.io~projected/kube-api-access-njltq:{mountpoint:/var/lib/kubelet/pods/746ec817-962d-4cb7-9767-0be957f2c533/volumes/kubernetes.io~projected/kube-api-access-njltq major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74a78648-f7a8-4363-9524-8bbc4b6a14a3/volumes/kubernetes.io~projected/kube-api-access-vq6rt:{mountpoint:/var/lib/kubelet/pods/74a78648-f7a8-4363-9524-8bbc4b6a14a3/volumes/kubernetes.io~projected/kube-api-access-vq6rt major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/75385577-f877-463f-a0fc-c9e2b844e840/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/75385577-f877-463f-a0fc-c9e2b844e840/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/89518dc7-2a05-4db9-b4f4-0662fd34d01b/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/89518dc7-2a05-4db9-b4f4-0662fd34d01b/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/89518dc7-2a05-4db9-b4f4-0662fd34d01b/volumes/kubernetes.io~projected/kube-api-access-679w9:{mountpoint:/var/lib/kubelet/pods/89518dc7-2a05-4db9-b4f4-0662fd34d01b/volumes/kubernetes.io~projected/kube-api-access-679w9 major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/89518dc7-2a05-4db9-b4f4-0662fd34d01b/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/89518dc7-2a05-4db9-b4f4-0662fd34d01b/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1938ddf-e712-417e-9741-8723017995b5/volumes/kubernetes.io~projected/kube-api-access-2xmmj:{mountpoint:/var/lib/kubelet/pods/c1938ddf-e712-417e-9741-8723017995b5/volumes/kubernetes.io~projected/kube-api-access-2xmmj major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e724db28-b13f-4bfa-89ba-98eb630e4241/volumes/kubernetes.io~projected/kube-api-access-h4lxq:{mountpoint:/var/lib/kubelet/pods/e724db28-b13f-4bfa-89ba-98eb630e4241/volumes/kubernetes.io~projected/kube-api-access-h4lxq major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e894be05-f132-4b29-a974-2ddf29c3e991/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/e894be05-f132-4b29-a974-2ddf29c3e991/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e894be05-f132-4b29-a974-2ddf29c3e991/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/e894be05-f132-4b29-a974-2ddf29c3e991/volumes/kubernetes.io~empty-dir/tmp major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e894be05-f132-4b29-a974-2ddf29c3e991/volumes/kubernetes.io~projected/kube-api-access-9tgcb:{mountpoint:/var/lib/kubelet/pods/e894be05-f132-4b29-a974-2ddf29c3e991/volumes/kubernetes.io~projected/kube-api-access-9tgcb major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eea4d604-1949-4ba4-964d-e57f74ce431a/volumes/kubernetes.io~projected/kube-api-access-67l68:{mountpoint:/var/lib/kubelet/pods/eea4d604-1949-4ba4-964d-e57f74ce431a/volumes/kubernetes.io~projected/kube-api-access-67l68 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eea4d604-1949-4ba4-964d-e57f74ce431a/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/eea4d604-1949-4ba4-964d-e57f74ce431a/volumes/kubernetes.io~secret/metrics-certs major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/faef07bf-622a-4f45-8e07-3a64933a6ef8/volumes/kubernetes.io~projected/kube-api-access-2hswr:{mountpoint:/var/lib/kubelet/pods/faef07bf-622a-4f45-8e07-3a64933a6ef8/volumes/kubernetes.io~projected/kube-api-access-2hswr major:0 minor:201 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/40da5cdf96b0fd894ece885637ed85d385c15655f5d965f36bea6edf94997a42/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/f89fc324de65c6a81d6491c5910ebffc5f6da599b4d490dd7c6b23e8096394e7/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/d442a63fdd2fb8f1c6f353e1991fe58e36e7a510698974eea091d416643fe3fb/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-105:{mountpoint:/var/lib/containers/storage/overlay/7d55a0e6e41a8e47c2225d62b7faf8404e360ff06df61598b5dafe46c2d8e320/merged major:0 minor:105 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/0e024cc4f7ea38de6c0bf8db42f7ec2d4e2e78734c62232d5c4f76d97206dccd/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/8a3820f4df88a7072473b305dee306204639ac7037fafdd1c16d05d1aa6425f8/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/473b41aefe4e5c2048bdef11d8b462069e40c96f28379664f7fa074163efa359/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/accfb4ab9ed8fe3e979d34af04cb3620055c4c57488f36998f4f578bc2a38a2b/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/800feb8f8a3692662310e35c298e44bc5503dee3cafe6613800499ef88c46406/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/d3628fae765bd5d55af52879258be9de6f3fe7390b10462c1585ec524376d775/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/9fd87480f5a3335290c762ec058e60755b295d02d62fda908d06a33f10d26fb2/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/aadc51c2c981b815e04349ba970c3292abcc73dbcc2d913145d30db3c8431452/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/d06ffbb9ac04ee4277a5d007e368c32a6b9c84da06875aaba667636d637f4845/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/6d639c2db1492c8724b3ecea3a6e7d5b3ff2238c85866bd08681a9a82eb98d59/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/933221d05fcf95664abc28fb7e248fedecd66511ca3929756c1193f25d357041/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/08ed5c889509067fd7b18db396a3bf8b386b19d2f0bcfa1c419239eb00ae6967/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/c201619a507de58f30d0e1171a24ab36f2d6f0c5d84c3447f0ee9981fc18f8c8/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/f715ebcd04bf7bf4b6bad41122f99e68767d9b8c7c51c89dde386a53e5888ca3/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/16ec919a9cb552650e7faa0a6020aedd5cffc077df2ce38df4012b2ee839cfff/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c1b83060d7673f358a4115743ab1f2c988c3cf43122b53f4f4feb6324eb0a108/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/055849f8c71a4a65e38a7e1729899131b7e98b9d27572e091ca649fab48b2dae/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-208:{mountpoint:/var/lib/containers/storage/overlay/ecf44418449a2c8dab24ad8cd76daa7f563e8e5f506dcc79eae500aa8981306b/merged major:0 minor:208 fsType:overlay blockSize:0} overlay_0-210:{mountpoint:/var/lib/containers/storage/overlay/672dcc54c0e41f06785f2dbd1ca44ecdce080fbec27ede7283d7088abae6f47c/merged major:0 minor:210 fsType:overlay blockSize:0} overlay_0-212:{mountpoint:/var/lib/containers/storage/overlay/62e33d60681ddf911f75f4a931debfc195f27b83f929883a4f434866fc089544/merged major:0 minor:212 fsType:overlay blockSize:0} overlay_0-214:{mountpoint:/var/lib/containers/storage/overlay/3904d7d0bd0b83454472d302c1949922fecdcedcb74b550bc1bbbb1bf2e5b4a9/merged major:0 minor:214 fsType:overlay blockSize:0} overlay_0-227:{mountpoint:/var/lib/containers/storage/overlay/75ba660d5d78d8fc976b743685266f26c2c3241228cf9cebfa41b856c44be2d2/merged major:0 minor:227 fsType:overlay blockSize:0} overlay_0-238:{mountpoint:/var/lib/containers/storage/overlay/70f5916437745c25260b48f24e35048eaa84ba741c186bfe486acf58937cc9df/merged major:0 minor:238 fsType:overlay blockSize:0} overlay_0-240:{mountpoint:/var/lib/containers/storage/overlay/46fe291708d465ee08c733321674b12c9b2abdeff249e6fbd8413ccb21d10bf6/merged major:0 minor:240 fsType:overlay blockSize:0} overlay_0-242:{mountpoint:/var/lib/containers/storage/overlay/699f864f93d930f2c54d380db8038768a7aac2ba1b8f03c78fc4b8a7b480ca61/merged major:0 minor:242 fsType:overlay blockSize:0} overlay_0-244:{mountpoint:/var/lib/containers/storage/overlay/53223455291f4a0b9edccaa0dc283086a8210c98db1596d7703b687585b79643/merged major:0 minor:244 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/8b19032bf1af854f23a1f09b43aad190b0bd8aedaf4dd74649a47cdf1c98efe7/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/1efd232f6e8681bfd2f4b7cc35f054cb3e7804685d998aac3e94fd4cd8122346/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/f931ad2d5f140a0cfbe546a07f11f77c2651b8eeaffa5c1ac29022b63ebd5078/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/28ca683270d1657acf218da1c9dc9bdbd01b6329877760f90678d3392c65aacc/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/d8e9ae50b1d729ecbe347939fb81254404d03dcacd1568dbfffa0bf564632638/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-62:{mountpoint:/var/lib/containers/storage/overlay/4ec12434a7d72721d7f0f8849fd73cc19c3d005baf3bd34e826d642a31e57d83/merged major:0 minor:62 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/851fe5942adbe4fa075b2e7c61ee9724774165822fd319e7dfd14c107e632ebd/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/ef74b9ce520d9f6f72eeb9e177e75af477678a2d393b50159ab8d5c06802cad4/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/b792f39bcc7e42f8deca346e42968fc72123a5e2029d279a0bdfe0265b064391/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/70b02598e09f409ccc4e3a3fafbb43aaf61ddcfe6bc2be2545d4a571a41f54c6/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/68b987f1eb37783582ad3a251c542026d088c5e6ccd1ea19195b7a76aaed6476/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/8752c5c1218defa23885ceebdbe864f15f11caf1a7f1490259fee952f2fc252b/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/50e35997f4eebb6923a7d63e28c1d6e1034e145050fb2a2922bb2de7b860f3fc/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/9fed1f3371186aab1ad827a57fa43092e53595b0d34b9c7292ec7d5b485123d8/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/791e1104a46ebc31b4cbe7e4b89cf0feec1b53a0ea4bc99c1cef531320bb5b89/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.654627    4585 manager.go:217] Machine: {Timestamp:2026-10-30 15:38:10.654154339 +0000 UTC m=+0.089758677 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107660 MemoryCapacity:33164328960 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ec75f85b724e63ecf0d8bb54d5c32 SystemUUID:ec2ec75f-85b7-24e6-3ecf-0d8bb54d5c32 BootID:66d660f4-3143-43b4-a611-4512ad219a51 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/65cf929f79d064f6edb5c94df150a1e80fb57287fb4f0cc05e0ce7a09f2bb201/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/89518dc7-2a05-4db9-b4f4-0662fd34d01b/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e894be05-f132-4b29-a974-2ddf29c3e991/volumes/kubernetes.io~projected/kube-api-access-9tgcb DeviceMajor:0 DeviceMinor:48 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/89518dc7-2a05-4db9-b4f4-0662fd34d01b/volumes/kubernetes.io~projected/kube-api-access-679w9 DeviceMajor:0 DeviceMinor:55 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/2d430d20-3538-482d-8588-a073be77dbd5/volumes/kubernetes.io~projected/kube-api-access-5qrm4 DeviceMajor:0 DeviceMinor:59 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8264aa0247ae7f6cbd5ab64e6bcb7311b4decb4154d8b89f2f30e7e0361f2a0e/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-210 DeviceMajor:0 DeviceMinor:210 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/048de475f8b290e82c77d99a0d039d251157580efe3b3a4b438f0e8474e01f14/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8/volumes/kubernetes.io~projected/kube-api-access-zmmnk DeviceMajor:0 DeviceMinor:56 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/beaf7b8db55c96a489424f1c922d6f37956faf14b0eb79c585d47aca26ba8017/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-240 DeviceMajor:0 DeviceMinor:240 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/75385577-f877-463f-a0fc-c9e2b844e840/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:229 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bf1499191d4923a8cb698ff7331daa7d998e88477d80296871135bdfbb9a65c3/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/74a78648-f7a8-4363-9524-8bbc4b6a14a3/volumes/kubernetes.io~projected/kube-api-access-vq6rt DeviceMajor:0 DeviceMinor:52 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/162edece99bb29c7a00db9fdadf2c50e8bdbc11b52679c503295942edb970da0/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3/volumes/kubernetes.io~projected/kube-api-access-mcrnd DeviceMajor:0 DeviceMinor:202 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/e894be05-f132-4b29-a974-2ddf29c3e991/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:50 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b7a376aef574bec6d3dfe258c9c25ba012dd05d31e973a78c2200b0583131ab5/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/803266f2222e88e04f82849afd6f7142cd2ce34d501ae4454c05a51fb740eb52/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-105 DeviceMajor:0 DeviceMinor:105 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e724db28-b13f-4bfa-89ba-98eb630e4241/volumes/kubernetes.io~projected/kube-api-access-h4lxq DeviceMajor:0 DeviceMinor:51 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/20a5f545-8ad7-45e5-ae97-1abfeb8d1080/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/20a5f545-8ad7-45e5-ae97-1abfeb8d1080/volumes/kubernetes.io~projected/kube-api-access-fmwx5 DeviceMajor:0 DeviceMinor:197 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-227 DeviceMajor:0 DeviceMinor:227 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eea4d604-1949-4ba4-964d-e57f74ce431a/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:230 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/92872077501cceac18b31f6e598a4ee0a7f4ca76db467b55a4d0082cb9a8fff0/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-208 DeviceMajor:0 DeviceMinor:208 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/24d34a460f8be378724d6aa245071044c917cb5b8ab8ede677bc2c4e57971a4a/userdata/shm DeviceMajor:0 DeviceMinor:225 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-242 DeviceMajor:0 DeviceMinor:242 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-244 DeviceMajor:0 DeviceMinor:244 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/89518dc7-2a05-4db9-b4f4-0662fd34d01b/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:54 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/94f3402f60b856d21c434e87a91fea567bc6a586c8c8599158c4ae596e5484db/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6a9a6ff0-5946-4635-9f97-c7be77c197c5/volumes/kubernetes.io~projected/kube-api-access-fpqs9 DeviceMajor:0 DeviceMinor:203 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/e894be05-f132-4b29-a974-2ddf29c3e991/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:49 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bdc96a647161c0911bf596f6315eb6a35515b5b4d24abf0392512ae47db743cf/userdata/shm DeviceMajor:0 DeviceMinor:204 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6a9a6ff0-5946-4635-9f97-c7be77c197c5/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:220 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/c1938ddf-e712-417e-9741-8723017995b5/volumes/kubernetes.io~projected/kube-api-access-2xmmj DeviceMajor:0 DeviceMinor:47 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4cc7d376-7849-4d50-8c0f-3128e474b54c/volumes/kubernetes.io~projected/kube-api-access-nrsng DeviceMajor:0 DeviceMinor:58 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/05854b46ac6e3336a8bed1009b92cc45799e3ffe9ad66d2312c3d3ac9d60e213/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9b9096edeb1ab5811232cdeaaf985cb99938bbbdec270bd2c89a7962a39cc45a/userdata/shm DeviceMajor:0 DeviceMinor:206 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/746ec817-962d-4cb7-9767-0be957f2c533/volumes/kubernetes.io~projected/kube-api-access-njltq DeviceMajor:0 DeviceMinor:231 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/eea4d604-1949-4ba4-964d-e57f74ce431a/volumes/kubernetes.io~projected/kube-api-access-67l68 DeviceMajor:0 DeviceMinor:57 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e416e74a75496c398348f99d57a4bf534ea1e1a0a8672a916e4ac8ade6b40f2e/userdata/shm DeviceMajor:0 DeviceMinor:64 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/faef07bf-622a-4f45-8e07-3a64933a6ef8/volumes/kubernetes.io~projected/kube-api-access-2hswr DeviceMajor:0 DeviceMinor:201 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-212 DeviceMajor:0 DeviceMinor:212 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/587ce512-7067-42a2-9cf1-2babd4feae72/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:53 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-62 DeviceMajor:0 DeviceMinor:62 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0f4bd13535d54ddaa5881bfebf22d7961425d6759fb312230057ec08e165bbc1/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-214 DeviceMajor:0 DeviceMinor:214 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-238 DeviceMajor:0 DeviceMinor:238 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:048de475f8b290e MacAddress:9e:ce:84:ca:38:00 Speed:10000 Mtu:8901} {Name:05854b46ac6e333 MacAddress:b2:31:f8:b1:aa:a5 Speed:10000 Mtu:8901} {Name:24d34a460f8be37 MacAddress:2a:c4:29:d1:88:35 Speed:10000 Mtu:8901} {Name:92872077501ccea MacAddress:ea:d8:7c:8b:e2:26 Speed:10000 Mtu:8901} {Name:9b9096edeb1ab58 MacAddress:56:53:fa:18:61:e4 Speed:10000 Mtu:8901} {Name:bdc96a647161c09 MacAddress:4e:48:d9:d1:b1:e2 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:d8:f0:f3:39 Speed:0 Mtu:9001} {Name:br-int MacAddress:36:c2:fa:30:2a:bb Speed:0 Mtu:8901} {Name:ens5 MacAddress:02:ff:d8:f0:f3:39 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:96:5c:d1:c9:37:c8 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:e2:8d:bd:7f:57:1f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164328960 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.654896    4585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.654941    4585 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.655452    4585 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.655501    4585 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-110.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.655810    4585 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.655824    4585 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.655844    4585 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.655862    4585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.655971    4585 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.656226    4585 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.656571    4585 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.656621    4585 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.656643    4585 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.656657    4585 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.656670    4585 kubelet.go:405] "Adding apiserver pod source"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.656681    4585 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.657292    4585 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.657302    4585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.657748    4585 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.659624    4585 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.659635    4585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.659732    4585 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.659906    4585 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.659932    4585 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660015    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660026    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660032    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660037    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660043    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660049    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660055    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660061    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660067    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660073    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660081    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660090    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660115    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.660121    4585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.663442    4585 server.go:1263] "Started kubelet"
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.663533    4585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.663595    4585 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.663650    4585 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 15:38:10.664717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.663907    4585 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 15:38:10.666152 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.665713    4585 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 15:38:10.667086 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.667068    4585 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 15:38:10.667143 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.667085    4585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 15:38:10.667187 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.667180    4585 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 15:38:10.667243 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.667191    4585 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 15:38:10.667296 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:10.667264    4585 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-110.ec2.internal\" not found"
Oct 30 15:38:10.667296 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.667288    4585 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 15:38:10.668002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.667152    4585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 15:32:33 +0000 UTC" deadline="2028-06-15 05:09:39.213248394 +0000 UTC"
Oct 30 15:38:10.668002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.667462    4585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14245h31m28.545791442s"
Oct 30 15:38:10.669042 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.668861    4585 factory.go:55] Registering systemd factory
Oct 30 15:38:10.669042 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.668886    4585 factory.go:223] Registration of the systemd container factory successfully
Oct 30 15:38:10.670196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.669796    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" volumeName="kubernetes.io/projected/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-kube-api-access-mcrnd" seLinuxMountContext=""
Oct 30 15:38:10.670196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.669846    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cc7d376-7849-4d50-8c0f-3128e474b54c" volumeName="kubernetes.io/empty-dir/4cc7d376-7849-4d50-8c0f-3128e474b54c-tmp-dir" seLinuxMountContext=""
Oct 30 15:38:10.670196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.669862    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8" volumeName="kubernetes.io/projected/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-kube-api-access-zmmnk" seLinuxMountContext=""
Oct 30 15:38:10.670196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.669880    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74a78648-f7a8-4363-9524-8bbc4b6a14a3" volumeName="kubernetes.io/projected/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kube-api-access-vq6rt" seLinuxMountContext=""
Oct 30 15:38:10.670196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.669984    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="89518dc7-2a05-4db9-b4f4-0662fd34d01b" volumeName="kubernetes.io/projected/89518dc7-2a05-4db9-b4f4-0662fd34d01b-kube-api-access-679w9" seLinuxMountContext=""
Oct 30 15:38:10.670196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.669997    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1938ddf-e712-417e-9741-8723017995b5" volumeName="kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-binary-copy" seLinuxMountContext=""
Oct 30 15:38:10.670196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670129    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1938ddf-e712-417e-9741-8723017995b5" volumeName="kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 15:38:10.670196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670143    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e894be05-f132-4b29-a974-2ddf29c3e991" volumeName="kubernetes.io/projected/e894be05-f132-4b29-a974-2ddf29c3e991-kube-api-access-9tgcb" seLinuxMountContext=""
Oct 30 15:38:10.670196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670168    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2d430d20-3538-482d-8588-a073be77dbd5" volumeName="kubernetes.io/projected/2d430d20-3538-482d-8588-a073be77dbd5-kube-api-access-5qrm4" seLinuxMountContext=""
Oct 30 15:38:10.670196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670182    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e724db28-b13f-4bfa-89ba-98eb630e4241" volumeName="kubernetes.io/projected/e724db28-b13f-4bfa-89ba-98eb630e4241-kube-api-access-h4lxq" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670200    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cc7d376-7849-4d50-8c0f-3128e474b54c" volumeName="kubernetes.io/projected/4cc7d376-7849-4d50-8c0f-3128e474b54c-kube-api-access-nrsng" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670332    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="587ce512-7067-42a2-9cf1-2babd4feae72" volumeName="kubernetes.io/configmap/587ce512-7067-42a2-9cf1-2babd4feae72-konnectivity-ca" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670367    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="75385577-f877-463f-a0fc-c9e2b844e840" volumeName="kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670389    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e894be05-f132-4b29-a974-2ddf29c3e991" volumeName="kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-etc-tuned" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670408    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="faef07bf-622a-4f45-8e07-3a64933a6ef8" volumeName="kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670425    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20a5f545-8ad7-45e5-ae97-1abfeb8d1080" volumeName="kubernetes.io/secret/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-cert" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670437    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a9a6ff0-5946-4635-9f97-c7be77c197c5" volumeName="kubernetes.io/configmap/6a9a6ff0-5946-4635-9f97-c7be77c197c5-config-volume" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670454    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="89518dc7-2a05-4db9-b4f4-0662fd34d01b" volumeName="kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-config" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670465    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eea4d604-1949-4ba4-964d-e57f74ce431a" volumeName="kubernetes.io/projected/eea4d604-1949-4ba4-964d-e57f74ce431a-kube-api-access-67l68" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670482    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" volumeName="kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-service-ca-bundle" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670495    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="746ec817-962d-4cb7-9767-0be957f2c533" volumeName="kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670508    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="89518dc7-2a05-4db9-b4f4-0662fd34d01b" volumeName="kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670524    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="89518dc7-2a05-4db9-b4f4-0662fd34d01b" volumeName="kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-env-overrides" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670536    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="faef07bf-622a-4f45-8e07-3a64933a6ef8" volumeName="kubernetes.io/projected/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-api-access-2hswr" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670553    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="587ce512-7067-42a2-9cf1-2babd4feae72" volumeName="kubernetes.io/secret/587ce512-7067-42a2-9cf1-2babd4feae72-agent-certs" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670565    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="faef07bf-622a-4f45-8e07-3a64933a6ef8" volumeName="kubernetes.io/empty-dir/faef07bf-622a-4f45-8e07-3a64933a6ef8-data-volume" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670578    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" volumeName="kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670595    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" volumeName="kubernetes.io/empty-dir/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-archives-path" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670620    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="20a5f545-8ad7-45e5-ae97-1abfeb8d1080" volumeName="kubernetes.io/projected/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-kube-api-access-fmwx5" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670633    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2d430d20-3538-482d-8588-a073be77dbd5" volumeName="kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-cni-binary-copy" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670644    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2d430d20-3538-482d-8588-a073be77dbd5" volumeName="kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-multus-daemon-config" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670661    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1938ddf-e712-417e-9741-8723017995b5" volumeName="kubernetes.io/projected/c1938ddf-e712-417e-9741-8723017995b5-kube-api-access-2xmmj" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670673    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e724db28-b13f-4bfa-89ba-98eb630e4241" volumeName="kubernetes.io/configmap/e724db28-b13f-4bfa-89ba-98eb630e4241-iptables-alerter-script" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670689    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eea4d604-1949-4ba4-964d-e57f74ce431a" volumeName="kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670742    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8" volumeName="kubernetes.io/configmap/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-serviceca" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670757    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a9a6ff0-5946-4635-9f97-c7be77c197c5" volumeName="kubernetes.io/empty-dir/6a9a6ff0-5946-4635-9f97-c7be77c197c5-tmp-dir" seLinuxMountContext=""
Oct 30 15:38:10.670758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670772    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a9a6ff0-5946-4635-9f97-c7be77c197c5" volumeName="kubernetes.io/projected/6a9a6ff0-5946-4635-9f97-c7be77c197c5-kube-api-access-fpqs9" seLinuxMountContext=""
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670786    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a9a6ff0-5946-4635-9f97-c7be77c197c5" volumeName="kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls" seLinuxMountContext=""
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670803    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="89518dc7-2a05-4db9-b4f4-0662fd34d01b" volumeName="kubernetes.io/secret/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670816    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1938ddf-e712-417e-9741-8723017995b5" volumeName="kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670923    4585 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e894be05-f132-4b29-a974-2ddf29c3e991" volumeName="kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-tmp" seLinuxMountContext=""
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670942    4585 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.670950    4585 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.671560    4585 factory.go:153] Registering CRI-O factory
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.671578    4585 factory.go:223] Registration of the crio container factory successfully
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.671633    4585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.671661    4585 factory.go:103] Registering Raw factory
Oct 30 15:38:10.671963 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.671679    4585 manager.go:1199] Started watching for new ooms in manager
Oct 30 15:38:10.672549 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.672342    4585 manager.go:319] Starting recovery of all containers
Oct 30 15:38:10.686050 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.686029    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:38:10.686783 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.686763    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:38:10.692598 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.692528    4585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 15:38:10.694061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.694036    4585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 15:38:10.694061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.694061    4585 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 15:38:10.694227 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.694084    4585 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 15:38:10.694227 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:10.694125    4585 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 15:38:10.707484 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.707462    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:38:10.707865 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.707840    4585 generic.go:356] "Generic (PLEG): container finished" podID="89518dc7-2a05-4db9-b4f4-0662fd34d01b" containerID="d3116e1f696328814f57df48db38e126c80066a4d1433013dc3e00a2c25440eb" exitCode=1
Oct 30 15:38:10.711295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.711268    4585 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="6193e207ac16eaacfb3a5bc26e29612cc90f02f9ac114e97f96a49638de2718e" exitCode=0
Oct 30 15:38:10.711392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.711298    4585 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="9f6875d39162d938468b01f6b96a5aed68716240a1cff29385dd52968d715697" exitCode=0
Oct 30 15:38:10.711392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.711309    4585 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="e8ce4f234049c8bdc891ea9142474842dea90017e71b9941e71358720cbe031a" exitCode=0
Oct 30 15:38:10.711392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.711318    4585 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="7ddf5be25a25cbd1f40899cbf264f8a2a53ec57c173218de42c2f929874b8659" exitCode=0
Oct 30 15:38:10.711392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.711325    4585 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="f094a52a17f9dffdb2fbc2dcc7bc6b759caff09d0821cc69df009e6e98a92ea6" exitCode=0
Oct 30 15:38:10.711392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.711333    4585 generic.go:356] "Generic (PLEG): container finished" podID="c1938ddf-e712-417e-9741-8723017995b5" containerID="1ece63077274e5c1d8c098df81ccd9a4f2d973d47dd45be5c26363fa45542672" exitCode=0
Oct 30 15:38:10.721481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.721457    4585 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:38:10.721672 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.721650    4585 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:38:10.722475 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.722454    4585 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 15:38:10.722601 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.722583    4585 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:38:10.732752 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.732726    4585 generic.go:356] "Generic (PLEG): container finished" podID="3cd74d1416e17adb1d88ce1b2d585255" containerID="f3b76182b402252f9326aedc0d88cadbf9721d8760d64c3fd0732c4f8036671b" exitCode=0
Oct 30 15:38:10.734583 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.734550    4585 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 15:38:10.746439 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.746419    4585 manager.go:324] Recovery completed
Oct 30 15:38:10.763538 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763473    4585 cpu_manager.go:226] "Starting" policy="none"
Oct 30 15:38:10.763538 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763489    4585 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 15:38:10.763538 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763510    4585 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 15:38:10.763716 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763636    4585 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 15:38:10.763716 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763647    4585 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 15:38:10.763716 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763666    4585 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 15:38:10.763716 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763672    4585 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 15:38:10.763716 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763677    4585 policy_none.go:50] "Start"
Oct 30 15:38:10.763716 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763686    4585 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 15:38:10.763716 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763695    4585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 15:38:10.763905 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763819    4585 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 15:38:10.763905 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763825    4585 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 15:38:10.763905 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.763831    4585 policy_none.go:44] "Start"
Oct 30 15:38:10.767884 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.767853    4585 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 15:38:10.770423 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.770408    4585 manager.go:341] "Starting Device Plugin manager"
Oct 30 15:38:10.770913 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:10.770434    4585 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 15:38:10.770953 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.770916    4585 server.go:87] "Starting device plugin registration server"
Oct 30 15:38:10.771188 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.771177    4585 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 15:38:10.771232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.771193    4585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 15:38:10.771315 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.771297    4585 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 15:38:10.771495 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.771433    4585 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 15:38:10.771495 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.771493    4585 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 15:38:10.771495 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.771499    4585 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 15:38:10.771701 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.771607    4585 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T15:38:10.771481774Z","UUID":"60881030-0905-4cb6-a71b-fc083320c339","Handler":null,"Name":"","Endpoint":""}
Oct 30 15:38:10.772736 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:10.772710    4585 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 15:38:10.773087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.773070    4585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 15:38:10.773171 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.773107    4585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 15:38:10.794222 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.794175    4585 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"]
Oct 30 15:38:10.794772 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.794735    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal" event={"ID":"50080b7deadb8e30dee000b7f0bbadfa","Type":"ContainerStarted","Data":"d165bcdee810ef3fdc18bb2ac44630fc8301beba40100cdc220623039309f356"}
Oct 30 15:38:10.794823 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.794788    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal" event={"ID":"50080b7deadb8e30dee000b7f0bbadfa","Type":"ContainerStarted","Data":"e416e74a75496c398348f99d57a4bf534ea1e1a0a8672a916e4ac8ade6b40f2e"}
Oct 30 15:38:10.794823 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.794797    4585 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:10.794905 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.794847    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal" event={"ID":"3cd74d1416e17adb1d88ce1b2d585255","Type":"ContainerStarted","Data":"64ac7c050efae956be088dd8ae6d0b213fd1de509b714a08e720ac7301a703bf"}
Oct 30 15:38:10.794905 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.794868    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal" event={"ID":"3cd74d1416e17adb1d88ce1b2d585255","Type":"ContainerDied","Data":"f3b76182b402252f9326aedc0d88cadbf9721d8760d64c3fd0732c4f8036671b"}
Oct 30 15:38:10.794905 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.794885    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal" event={"ID":"3cd74d1416e17adb1d88ce1b2d585255","Type":"ContainerStarted","Data":"bf1499191d4923a8cb698ff7331daa7d998e88477d80296871135bdfbb9a65c3"}
Oct 30 15:38:10.794993 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.794932    4585 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:10.817786 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.817760    4585 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 15:38:10.817921 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:10.817817    4585 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-110.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:10.818257 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.818241    4585 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 15:38:10.818318 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:10.818279    4585 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:10.872127 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.872096    4585 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 15:38:10.873438 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.873413    4585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 15:38:10.873556 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.873452    4585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 15:38:10.873556 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.873466    4585 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-110.ec2.internal" event="NodeHasSufficientPID"
Oct 30 15:38:10.873556 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.873499    4585 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-110.ec2.internal"
Oct 30 15:38:10.907923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.907897    4585 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-140-110.ec2.internal"
Oct 30 15:38:10.908044 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.907961    4585 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-110.ec2.internal"
Oct 30 15:38:10.923727 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.923697    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:10.923838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.923733    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:10.923838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:10.923751    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50080b7deadb8e30dee000b7f0bbadfa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-110.ec2.internal\" (UID: \"50080b7deadb8e30dee000b7f0bbadfa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.024967 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.024884    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.024967 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.024907    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.024967 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.024961    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50080b7deadb8e30dee000b7f0bbadfa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-110.ec2.internal\" (UID: \"50080b7deadb8e30dee000b7f0bbadfa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.025341 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.024978    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.025341 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.025016    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3cd74d1416e17adb1d88ce1b2d585255-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" (UID: \"3cd74d1416e17adb1d88ce1b2d585255\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.025341 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.025056    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50080b7deadb8e30dee000b7f0bbadfa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-110.ec2.internal\" (UID: \"50080b7deadb8e30dee000b7f0bbadfa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.657507 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.657464    4585 apiserver.go:52] "Watching apiserver"
Oct 30 15:38:11.697670 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.697641    4585 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 15:38:11.698184 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.698161    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-xrrf9","kube-system/konnectivity-agent-trlcx","openshift-dns/dns-default-bxct5","openshift-ovn-kubernetes/ovnkube-node-sg7k4","openshift-insights/insights-runtime-extractor-zknrn","openshift-network-operator/iptables-alerter-zxvbz","kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal","openshift-insights/periodic-gathering-rssdr-2wg26","openshift-multus/multus-additional-cni-plugins-9xhq8","openshift-network-diagnostics/network-check-target-ttqnr","openshift-ingress-canary/ingress-canary-k42cc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal","openshift-multus/network-metrics-daemon-fqdrs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj","kube-system/global-pull-secret-syncer-9l9gb","openshift-cluster-node-tuning-operator/tuned-4wfrg","openshift-dns/node-resolver-ngvd2","openshift-multus/multus-lw4vm"]
Oct 30 15:38:11.706544 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.706518    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.714216 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714184    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 15:38:11.714414 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714398    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 15:38:11.714574 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714556    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-khtmm\""
Oct 30 15:38:11.714673 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714571    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 15:38:11.714673 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714580    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 15:38:11.714673 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714593    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-84s7k\""
Oct 30 15:38:11.714673 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714614    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 15:38:11.714673 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714643    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qtl5k\""
Oct 30 15:38:11.714673 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714660    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5ljdx\""
Oct 30 15:38:11.714673 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714659    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 15:38:11.714883 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.714691    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-xbwc7\""
Oct 30 15:38:11.715846 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.715826    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 15:38:11.715917 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.715904    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 15:38:11.716375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.716341    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 15:38:11.716646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.716625    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 15:38:11.716760 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.716714    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 15:38:11.716760 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.716717    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 15:38:11.716760 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.716756    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 15:38:11.716911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.716788    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 15:38:11.716911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.716826    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 15:38:11.716911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.716856    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 15:38:11.717007 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.716993    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 15:38:11.717062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.717053    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 15:38:11.719254 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.719238    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fmlw7\""
Oct 30 15:38:11.723428 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.723414    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ck89s\""
Oct 30 15:38:11.736145 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.736128    4585 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.736277 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.736265    4585 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.758797 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.758772    4585 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 15:38:11.758920 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:11.758816    4585 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-110.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.759367 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.759337    4585 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 15:38:11.759399 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:11.759389    4585 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-110.ec2.internal"
Oct 30 15:38:11.764371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764334    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 15:38:11.764451 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764334    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 15:38:11.764451 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764423    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 15:38:11.764540 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764454    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 15:38:11.764540 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764453    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 15:38:11.764540 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764492    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 15:38:11.764707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764688    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 15:38:11.764779 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764725    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 15:38:11.764779 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764729    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 15:38:11.765027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764821    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 15:38:11.765027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764876    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 15:38:11.765027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.764877    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 15:38:11.765027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.765003    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 15:38:11.765195 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.765036    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 15:38:11.765195 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.765073    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 15:38:11.765195 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.765106    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 15:38:11.765542 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.765529    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 15:38:11.768852 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.768837    4585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 15:38:11.771427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.771408    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 15:38:11.814398 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814337    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5dknm\""
Oct 30 15:38:11.814546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814335    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zt274\""
Oct 30 15:38:11.814546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814371    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mw526\""
Oct 30 15:38:11.814546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814480    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 15:38:11.814546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814519    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 15:38:11.814774 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814759    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rf9nq\""
Oct 30 15:38:11.814832 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814786    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-nlpvl\""
Oct 30 15:38:11.814832 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814807    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 15:38:11.814832 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814821    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mhkxg\""
Oct 30 15:38:11.815003 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814843    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8vjmf\""
Oct 30 15:38:11.815003 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814844    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 15:38:11.815003 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814820    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 15:38:11.815003 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814882    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ftvkm\""
Oct 30 15:38:11.815003 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.814952    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9c74v\""
Oct 30 15:38:11.815138 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.815020    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 15:38:11.841747 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.841720    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.841747 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.841748    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.841923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.841772    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-archives-path\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:11.841923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.841790    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-sys\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.841923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.841858    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-hostroot\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.842049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.841933    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-cnibin\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.842049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.841964    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-archives-path\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:11.842049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.841989    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.842193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842050    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysconfig\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.842193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842095    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-systemd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.842193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842127    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-env-overrides\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.842193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842178    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/587ce512-7067-42a2-9cf1-2babd4feae72-agent-certs\") pod \"konnectivity-agent-trlcx\" (UID: \"587ce512-7067-42a2-9cf1-2babd4feae72\") " pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:38:11.842392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842208    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:38:11.842392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842232    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-system-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.842392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842258    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-socket-dir-parent\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.842392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842322    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-trusted-ca-bundle\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:11.842392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842322    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.842392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842368    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-env-overrides\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.842392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842381    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-cnibin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.842631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842407    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-netns\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.842631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842423    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-bin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.842631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842438    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-etc-kubernetes\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.842631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842464    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a9a6ff0-5946-4635-9f97-c7be77c197c5-tmp-dir\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:11.842631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842501    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eea4d604-1949-4ba4-964d-e57f74ce431a-metrics-certs\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:38:11.842631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842552    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcrnd\" (UniqueName: \"kubernetes.io/projected/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-kube-api-access-mcrnd\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:11.842631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842584    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/587ce512-7067-42a2-9cf1-2babd4feae72-agent-certs\") pod \"konnectivity-agent-trlcx\" (UID: \"587ce512-7067-42a2-9cf1-2babd4feae72\") " pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:38:11.842631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842586    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-run\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.842631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842596    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a9a6ff0-5946-4635-9f97-c7be77c197c5-tmp-dir\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:11.842631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842620    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-node-log\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.842981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842693    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-trusted-ca-bundle\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:11.842981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842730    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e724db28-b13f-4bfa-89ba-98eb630e4241-host-slash\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:38:11.842981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842760    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.842981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842779    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.842981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842823    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-script-lib\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.842981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842861    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-sys-fs\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.842981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842887    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmmnk\" (UniqueName: \"kubernetes.io/projected/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-kube-api-access-zmmnk\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:38:11.842981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842913    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-conf-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.842981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.842956    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-service-ca-bundle\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:11.843295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843022    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-var-lib-kubelet\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.843295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843032    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-script-lib\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.843295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843135    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-ovn\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.843295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843163    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-device-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.843295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843188    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-serviceca\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:38:11.843295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843203    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-systemd-units\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.843295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843220    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-cni-binary-copy\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.843295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843246    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-k8s-cni-cncf-io\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.843295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843270    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc7d376-7849-4d50-8c0f-3128e474b54c-hosts-file\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:38:11.843295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843294    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc7d376-7849-4d50-8c0f-3128e474b54c-tmp-dir\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843389    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-kubelet-config\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843399    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc7d376-7849-4d50-8c0f-3128e474b54c-tmp-dir\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843438    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-systemd\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843485    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-tmp\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843497    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-cni-binary-copy\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843509    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843534    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-system-cni-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843545    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-service-ca-bundle\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843556    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-tmp\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843562    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/faef07bf-622a-4f45-8e07-3a64933a6ef8-crio-socket\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.843632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843594    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hswr\" (UniqueName: \"kubernetes.io/projected/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-api-access-2hswr\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.843947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843648    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-kubernetes\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.843947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843750    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-conf\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.843947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843804    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.843947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843830    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-socket-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.843947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843847    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4lxq\" (UniqueName: \"kubernetes.io/projected/e724db28-b13f-4bfa-89ba-98eb630e4241-kube-api-access-h4lxq\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:38:11.843947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843885    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-host\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:38:11.843947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843886    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-serviceca\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:38:11.843947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843926    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/faef07bf-622a-4f45-8e07-3a64933a6ef8-data-volume\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.844172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.843991    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.844172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844022    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/faef07bf-622a-4f45-8e07-3a64933a6ef8-data-volume\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.844172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844051    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67l68\" (UniqueName: \"kubernetes.io/projected/eea4d604-1949-4ba4-964d-e57f74ce431a-kube-api-access-67l68\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:38:11.844172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844069    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-multus-daemon-config\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.844172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844087    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-multus-certs\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.844172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844164    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-dbus\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:11.844397 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844241    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:11.844397 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844286    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d430d20-3538-482d-8588-a073be77dbd5-multus-daemon-config\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.844397 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844306    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpqs9\" (UniqueName: \"kubernetes.io/projected/6a9a6ff0-5946-4635-9f97-c7be77c197c5-kube-api-access-fpqs9\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:11.844397 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844335    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-kubelet\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.844397 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844383    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-slash\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.844550 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844400    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-bin\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.844550 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844417    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.844550 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844435    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovn-node-metrics-cert\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.844550 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844451    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a9a6ff0-5946-4635-9f97-c7be77c197c5-metrics-tls\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:11.844550 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844520    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.844707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844584    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.844707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844612    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-etc-tuned\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.844707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844656    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-netd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.844707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844684    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-config\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.844707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844689    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e894be05-f132-4b29-a974-2ddf29c3e991-etc-tuned\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.844859 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844714    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:11.844859 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844745    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.844859 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844772    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-etc-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.844859 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844796    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-registration-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.844859 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844835    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovn-node-metrics-cert\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.845000 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844888    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq6rt\" (UniqueName: \"kubernetes.io/projected/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kube-api-access-vq6rt\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.845000 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844895    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89518dc7-2a05-4db9-b4f4-0662fd34d01b-ovnkube-config\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.845000 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844970    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75385577-f877-463f-a0fc-c9e2b844e840-original-pull-secret\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:11.845000 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.844988    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e724db28-b13f-4bfa-89ba-98eb630e4241-iptables-alerter-script\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:38:11.845112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845017    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-kubelet\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.845112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845042    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrsng\" (UniqueName: \"kubernetes.io/projected/4cc7d376-7849-4d50-8c0f-3128e474b54c-kube-api-access-nrsng\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:38:11.845112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845075    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xmmj\" (UniqueName: \"kubernetes.io/projected/c1938ddf-e712-417e-9741-8723017995b5-kube-api-access-2xmmj\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.845191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845111    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmwx5\" (UniqueName: \"kubernetes.io/projected/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-kube-api-access-fmwx5\") pod \"ingress-canary-k42cc\" (UID: \"20a5f545-8ad7-45e5-ae97-1abfeb8d1080\") " pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:11.845191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845161    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-host\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.845191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845180    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tgcb\" (UniqueName: \"kubernetes.io/projected/e894be05-f132-4b29-a974-2ddf29c3e991-kube-api-access-9tgcb\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.845301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845195    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-var-lib-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.845301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845216    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.845301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845243    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679w9\" (UniqueName: \"kubernetes.io/projected/89518dc7-2a05-4db9-b4f4-0662fd34d01b-kube-api-access-679w9\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.845301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845266    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e724db28-b13f-4bfa-89ba-98eb630e4241-iptables-alerter-script\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:38:11.845301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845272    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.845547 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845306    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9a6ff0-5946-4635-9f97-c7be77c197c5-config-volume\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:11.845547 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845336    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-cert\") pod \"ingress-canary-k42cc\" (UID: \"20a5f545-8ad7-45e5-ae97-1abfeb8d1080\") " pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:11.845547 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845368    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:38:11.845547 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845466    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-modprobe-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.845547 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845500    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-netns\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.845547 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845529    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/587ce512-7067-42a2-9cf1-2babd4feae72-konnectivity-ca\") pod \"konnectivity-agent-trlcx\" (UID: \"587ce512-7067-42a2-9cf1-2babd4feae72\") " pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:38:11.845740 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845552    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-cert\") pod \"ingress-canary-k42cc\" (UID: \"20a5f545-8ad7-45e5-ae97-1abfeb8d1080\") " pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:11.845740 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845555    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a9a6ff0-5946-4635-9f97-c7be77c197c5-config-volume\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:11.845740 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845556    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-multus\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.845740 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845668    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qrm4\" (UniqueName: \"kubernetes.io/projected/2d430d20-3538-482d-8588-a073be77dbd5-kube-api-access-5qrm4\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.845740 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845712    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/587ce512-7067-42a2-9cf1-2babd4feae72-konnectivity-ca\") pod \"konnectivity-agent-trlcx\" (UID: \"587ce512-7067-42a2-9cf1-2babd4feae72\") " pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:38:11.845887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845743    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.845887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845763    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-lib-modules\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.845887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845778    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-log-socket\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.845989 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.845929    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c1938ddf-e712-417e-9741-8723017995b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.868453 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.868425    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hswr\" (UniqueName: \"kubernetes.io/projected/faef07bf-622a-4f45-8e07-3a64933a6ef8-kube-api-access-2hswr\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.868817 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.868795    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-679w9\" (UniqueName: \"kubernetes.io/projected/89518dc7-2a05-4db9-b4f4-0662fd34d01b-kube-api-access-679w9\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.868937 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.868899    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4lxq\" (UniqueName: \"kubernetes.io/projected/e724db28-b13f-4bfa-89ba-98eb630e4241-kube-api-access-h4lxq\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:38:11.868937 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.868899    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qrm4\" (UniqueName: \"kubernetes.io/projected/2d430d20-3538-482d-8588-a073be77dbd5-kube-api-access-5qrm4\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.869060 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.868971    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq6rt\" (UniqueName: \"kubernetes.io/projected/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kube-api-access-vq6rt\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.869060 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.868975    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrsng\" (UniqueName: \"kubernetes.io/projected/4cc7d376-7849-4d50-8c0f-3128e474b54c-kube-api-access-nrsng\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:38:11.869174 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.869156    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njltq\" (UniqueName: \"kubernetes.io/projected/746ec817-962d-4cb7-9767-0be957f2c533-kube-api-access-njltq\") pod \"network-check-target-ttqnr\" (UID: \"746ec817-962d-4cb7-9767-0be957f2c533\") " pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:38:11.869224 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.869186    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xmmj\" (UniqueName: \"kubernetes.io/projected/c1938ddf-e712-417e-9741-8723017995b5-kube-api-access-2xmmj\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.869593 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.869574    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67l68\" (UniqueName: \"kubernetes.io/projected/eea4d604-1949-4ba4-964d-e57f74ce431a-kube-api-access-67l68\") pod \"network-metrics-daemon-fqdrs\" (UID: \"eea4d604-1949-4ba4-964d-e57f74ce431a\") " pod="openshift-multus/network-metrics-daemon-fqdrs"
Oct 30 15:38:11.869654 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.869618    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpqs9\" (UniqueName: \"kubernetes.io/projected/6a9a6ff0-5946-4635-9f97-c7be77c197c5-kube-api-access-fpqs9\") pod \"dns-default-bxct5\" (UID: \"6a9a6ff0-5946-4635-9f97-c7be77c197c5\") " pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:11.869654 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.869628    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmmnk\" (UniqueName: \"kubernetes.io/projected/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-kube-api-access-zmmnk\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:38:11.869928 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.869899    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tgcb\" (UniqueName: \"kubernetes.io/projected/e894be05-f132-4b29-a974-2ddf29c3e991-kube-api-access-9tgcb\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.870046 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.870029    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmwx5\" (UniqueName: \"kubernetes.io/projected/20a5f545-8ad7-45e5-ae97-1abfeb8d1080-kube-api-access-fmwx5\") pod \"ingress-canary-k42cc\" (UID: \"20a5f545-8ad7-45e5-ae97-1abfeb8d1080\") " pod="openshift-ingress-canary/ingress-canary-k42cc"
Oct 30 15:38:11.873992 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.873975    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcrnd\" (UniqueName: \"kubernetes.io/projected/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-kube-api-access-mcrnd\") pod \"periodic-gathering-rssdr-2wg26\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") " pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:11.946518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946432    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-cnibin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.946518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946466    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-netns\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.946518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946481    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-bin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.946518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946495    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-etc-kubernetes\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.946518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946518    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-run\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946541    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-node-log\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946561    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-netns\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946584    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-cnibin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946565    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e724db28-b13f-4bfa-89ba-98eb630e4241-host-slash\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946596    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e724db28-b13f-4bfa-89ba-98eb630e4241-host-slash\") pod \"iptables-alerter-zxvbz\" (UID: \"e724db28-b13f-4bfa-89ba-98eb630e4241\") " pod="openshift-network-operator/iptables-alerter-zxvbz"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946609    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-run\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946619    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-node-log\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946569    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-bin\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946630    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946584    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-etc-kubernetes\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946675    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946705    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-sys-fs\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946735    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-conf-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946759    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-sys-fs\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946816    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-var-lib-kubelet\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946842    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-ovn\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946850    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-conf-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.946870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946867    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-device-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946886    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-var-lib-kubelet\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946895    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-systemd-units\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946871    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946908    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-device-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946897    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-ovn\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946923    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-k8s-cni-cncf-io\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946927    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-systemd-units\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946946    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc7d376-7849-4d50-8c0f-3128e474b54c-hosts-file\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946956    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-k8s-cni-cncf-io\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.946962    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-kubelet-config\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947008    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-kubelet-config\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947017    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-systemd\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947027    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc7d376-7849-4d50-8c0f-3128e474b54c-hosts-file\") pod \"node-resolver-ngvd2\" (UID: \"4cc7d376-7849-4d50-8c0f-3128e474b54c\") " pod="openshift-dns/node-resolver-ngvd2"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947043    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947060    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-system-cni-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947015    4585 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947072    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-systemd\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947079    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/faef07bf-622a-4f45-8e07-3a64933a6ef8-crio-socket\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947098    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947124    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-kubernetes\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947151    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-conf\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947166    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-kubernetes\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947125    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-system-cni-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947125    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/faef07bf-622a-4f45-8e07-3a64933a6ef8-crio-socket\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947175    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947211    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947223    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-socket-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947262    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-host\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947286    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysctl-conf\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947285    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947318    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947328    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-multus-certs\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947362    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-dbus\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947373    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-socket-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947384    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-kubelet\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947376    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-run-multus-certs\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947411    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-slash\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947423    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75385577-f877-463f-a0fc-c9e2b844e840-dbus\") pod \"global-pull-secret-syncer-9l9gb\" (UID: \"75385577-f877-463f-a0fc-c9e2b844e840\") " pod="kube-system/global-pull-secret-syncer-9l9gb"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947428    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-kubelet\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947329    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8-host\") pod \"node-ca-xrrf9\" (UID: \"630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8\") " pod="openshift-image-registry/node-ca-xrrf9"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947441    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-slash\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947449    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-bin\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947477    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-netd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947490    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-bin\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947500    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-etc-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947522    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-registration-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947530    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-cni-netd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947556    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-kubelet\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947572    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-etc-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947578    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-host\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947613    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-registration-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947616    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-var-lib-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.947611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947636    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-kubelet\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947648    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-var-lib-openvswitch\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947650    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947680    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947685    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-host\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947682    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947718    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-modprobe-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947724    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947741    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-netns\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947788    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-modprobe-d\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947826    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-host-run-netns\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947853    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-multus\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947889    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-host-var-lib-cni-multus\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947902    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-lib-modules\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947939    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-log-socket\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.947962    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948011    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-log-socket\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948021    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74a78648-f7a8-4363-9524-8bbc4b6a14a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-mvzgj\" (UID: \"74a78648-f7a8-4363-9524-8bbc4b6a14a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvzgj"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948029    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-lib-modules\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948048    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948074    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-sys\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948097    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-hostroot\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948119    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-cnibin\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948127    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948150    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-hostroot\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948154    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-sys\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948146    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysconfig\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948188    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-systemd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948192    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e894be05-f132-4b29-a974-2ddf29c3e991-etc-sysconfig\") pod \"tuned-4wfrg\" (UID: \"e894be05-f132-4b29-a974-2ddf29c3e991\") " pod="openshift-cluster-node-tuning-operator/tuned-4wfrg"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948154    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c1938ddf-e712-417e-9741-8723017995b5-cnibin\") pod \"multus-additional-cni-plugins-9xhq8\" (UID: \"c1938ddf-e712-417e-9741-8723017995b5\") " pod="openshift-multus/multus-additional-cni-plugins-9xhq8"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948214    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-system-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948236    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89518dc7-2a05-4db9-b4f4-0662fd34d01b-run-systemd\") pod \"ovnkube-node-sg7k4\" (UID: \"89518dc7-2a05-4db9-b4f4-0662fd34d01b\") " pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948239    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-socket-dir-parent\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948308    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-multus-socket-dir-parent\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.949253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.948320    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d430d20-3538-482d-8588-a073be77dbd5-system-cni-dir\") pod \"multus-lw4vm\" (UID: \"2d430d20-3538-482d-8588-a073be77dbd5\") " pod="openshift-multus/multus-lw4vm"
Oct 30 15:38:11.950283 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:11.949374    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/faef07bf-622a-4f45-8e07-3a64933a6ef8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zknrn\" (UID: \"faef07bf-622a-4f45-8e07-3a64933a6ef8\") " pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:12.006868 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:12.006835    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zknrn"
Oct 30 15:38:12.361503 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:12.361424    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zknrn"]
Oct 30 15:38:12.367891 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:12.367859    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaef07bf_622a_4f45_8e07_3a64933a6ef8.slice/crio-9c1e640bbd12d8aa2894f1df027143276672d19dacde751e10166ecfed628d13 WatchSource:0}: Error finding container 9c1e640bbd12d8aa2894f1df027143276672d19dacde751e10166ecfed628d13: Status 404 returned error can't find the container with id 9c1e640bbd12d8aa2894f1df027143276672d19dacde751e10166ecfed628d13
Oct 30 15:38:12.469679 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:12.469652    4585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 15:38:12.740190 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:12.740148    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zknrn" event={"ID":"faef07bf-622a-4f45-8e07-3a64933a6ef8","Type":"ContainerStarted","Data":"272c4ead9bf38ebe58ecee069e90936ff844871e96d20f458c4f85062688e199"}
Oct 30 15:38:12.740576 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:12.740213    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zknrn" event={"ID":"faef07bf-622a-4f45-8e07-3a64933a6ef8","Type":"ContainerStarted","Data":"9c1e640bbd12d8aa2894f1df027143276672d19dacde751e10166ecfed628d13"}
Oct 30 15:38:13.744803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:13.744724    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zknrn" event={"ID":"faef07bf-622a-4f45-8e07-3a64933a6ef8","Type":"ContainerStarted","Data":"ec3cbd8e1f1502a8a89033695d6b57838bea9a8f9732805956aa520a204f5eab"}
Oct 30 15:38:14.443458 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:14.443425    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:14.631527 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:14.631487    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:15.755821 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:15.755787    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zknrn" event={"ID":"faef07bf-622a-4f45-8e07-3a64933a6ef8","Type":"ContainerStarted","Data":"0190e30ea34e96c1720b6250a4653a56e4cf05f676e8f20c7c4c4c0620419684"}
Oct 30 15:38:17.343904 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:17.343878    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:17.344416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:17.344079    4585 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 15:38:17.387951 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:17.387914    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:38:17.445873 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:17.445839    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bxct5"
Oct 30 15:38:17.958821 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:17.958788    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:38:18.715417 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.715383    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-2zkc5-w5tp6"]
Oct 30 15:38:18.717617 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.717594    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.746431 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.746310    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-2zkc5-w5tp6"]
Oct 30 15:38:18.888543 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.888500    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-service-ca-bundle\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.888543 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.888547    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-trusted-ca-bundle\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.888787 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.888640    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e275df6d-03bb-4d35-9b50-f8b5ea98e226-archives-path\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.888787 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.888674    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwg82\" (UniqueName: \"kubernetes.io/projected/e275df6d-03bb-4d35-9b50-f8b5ea98e226-kube-api-access-vwg82\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.933161 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.933130    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d489c8c55-5ptpf"]
Oct 30 15:38:18.934893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.934877    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:18.986936 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.986749    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d489c8c55-5ptpf"]
Oct 30 15:38:18.988288 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.987153    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 15:38:18.989297 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.989270    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-trusted-ca-bundle\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.989427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.989328    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e275df6d-03bb-4d35-9b50-f8b5ea98e226-archives-path\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.989427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.989382    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwg82\" (UniqueName: \"kubernetes.io/projected/e275df6d-03bb-4d35-9b50-f8b5ea98e226-kube-api-access-vwg82\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.989540 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.989434    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-service-ca-bundle\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.989718 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.989689    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e275df6d-03bb-4d35-9b50-f8b5ea98e226-archives-path\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.990073 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.990051    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-service-ca-bundle\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.990246 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.990226    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-trusted-ca-bundle\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:18.994261 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.994243    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 15:38:18.994404 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.994393    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 15:38:18.994471 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.994457    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5wgs2\""
Oct 30 15:38:18.994471 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:18.994463    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 15:38:19.016006 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.015982    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwg82\" (UniqueName: \"kubernetes.io/projected/e275df6d-03bb-4d35-9b50-f8b5ea98e226-kube-api-access-vwg82\") pod \"periodic-gathering-2zkc5-w5tp6\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") " pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:19.030664 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.030642    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:19.090795 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.090768    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-image-registry-private-configuration\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.090912 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.090800    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlq7z\" (UniqueName: \"kubernetes.io/projected/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-kube-api-access-xlq7z\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.090912 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.090822    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-registry-certificates\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.090912 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.090840    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-ca-trust-extracted\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.091023 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.090915    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-installation-pull-secrets\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.091057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.091040    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-registry-tls\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.091098 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.091074    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-bound-sa-token\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.091156 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.091141    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-trusted-ca\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.191634 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.191594    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-trusted-ca\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.191789 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.191648    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-image-registry-private-configuration\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.191789 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.191667    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlq7z\" (UniqueName: \"kubernetes.io/projected/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-kube-api-access-xlq7z\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.191789 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.191683    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-registry-certificates\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.191789 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.191700    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-ca-trust-extracted\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.191789 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.191716    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-installation-pull-secrets\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.192048 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.191864    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-registry-tls\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.192048 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.191908    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-bound-sa-token\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.192578 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.192547    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-registry-certificates\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.192705 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.192644    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-ca-trust-extracted\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.192705 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.192682    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-trusted-ca\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.194661 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.194635    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-registry-tls\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.195052 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.195033    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-image-registry-private-configuration\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.195092 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.195046    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-installation-pull-secrets\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.217785 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.217761    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-bound-sa-token\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.217882 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.217817    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlq7z\" (UniqueName: \"kubernetes.io/projected/7a4454c5-1cc0-40d9-8e74-6adaf667fa66-kube-api-access-xlq7z\") pod \"image-registry-7d489c8c55-5ptpf\" (UID: \"7a4454c5-1cc0-40d9-8e74-6adaf667fa66\") " pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.244740 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.244678    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:19.340380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.340334    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-2zkc5-w5tp6"]
Oct 30 15:38:19.343501 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:19.343473    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode275df6d_03bb_4d35_9b50_f8b5ea98e226.slice/crio-279c51ea331e8569b31b535e1ac8d6a901efefe06e58e868d0ba0d88f5e7c100 WatchSource:0}: Error finding container 279c51ea331e8569b31b535e1ac8d6a901efefe06e58e868d0ba0d88f5e7c100: Status 404 returned error can't find the container with id 279c51ea331e8569b31b535e1ac8d6a901efefe06e58e868d0ba0d88f5e7c100
Oct 30 15:38:19.548049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.547969    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d489c8c55-5ptpf"]
Oct 30 15:38:19.550867 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:19.550836    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a4454c5_1cc0_40d9_8e74_6adaf667fa66.slice/crio-d7ea9fad1cde04ab8970fde532e75c92331eac06a4b13d91d1fce202f0c456f6 WatchSource:0}: Error finding container d7ea9fad1cde04ab8970fde532e75c92331eac06a4b13d91d1fce202f0c456f6: Status 404 returned error can't find the container with id d7ea9fad1cde04ab8970fde532e75c92331eac06a4b13d91d1fce202f0c456f6
Oct 30 15:38:19.771629 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.771591    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-2zkc5-w5tp6" event={"ID":"e275df6d-03bb-4d35-9b50-f8b5ea98e226","Type":"ContainerStarted","Data":"65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5"}
Oct 30 15:38:19.772098 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.771661    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-2zkc5-w5tp6" event={"ID":"e275df6d-03bb-4d35-9b50-f8b5ea98e226","Type":"ContainerStarted","Data":"279c51ea331e8569b31b535e1ac8d6a901efefe06e58e868d0ba0d88f5e7c100"}
Oct 30 15:38:19.773242 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.773215    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf" event={"ID":"7a4454c5-1cc0-40d9-8e74-6adaf667fa66","Type":"ContainerStarted","Data":"13068dc155ad88b2d8534f3f28bee8615a9b4b9b923a58e3dd08954e7d39a35b"}
Oct 30 15:38:19.773392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.773261    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf" event={"ID":"7a4454c5-1cc0-40d9-8e74-6adaf667fa66","Type":"ContainerStarted","Data":"d7ea9fad1cde04ab8970fde532e75c92331eac06a4b13d91d1fce202f0c456f6"}
Oct 30 15:38:19.816773 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.816376    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-2zkc5-w5tp6" podStartSLOduration=1.816335895 podStartE2EDuration="1.816335895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:19.816061232 +0000 UTC m=+9.251665581" watchObservedRunningTime="2026-10-30 15:38:19.816335895 +0000 UTC m=+9.251940245"
Oct 30 15:38:19.861677 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:19.861619    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf" podStartSLOduration=1.8616016690000001 podStartE2EDuration="1.861601669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:19.860827354 +0000 UTC m=+9.296431704" watchObservedRunningTime="2026-10-30 15:38:19.861601669 +0000 UTC m=+9.297206011"
Oct 30 15:38:20.776258 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:20.776230    4585 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 15:38:20.856631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:20.856602    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bxct5_6a9a6ff0-5946-4635-9f97-c7be77c197c5/dns/0.log"
Oct 30 15:38:20.992912 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:20.992848    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bxct5_6a9a6ff0-5946-4635-9f97-c7be77c197c5/kube-rbac-proxy/0.log"
Oct 30 15:38:21.992695 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:21.992656    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngvd2_4cc7d376-7849-4d50-8c0f-3128e474b54c/dns-node-resolver/0.log"
Oct 30 15:38:22.192478 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:22.192441    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7d489c8c55-5ptpf_7a4454c5-1cc0-40d9-8e74-6adaf667fa66/registry/0.log"
Oct 30 15:38:23.192537 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:23.192510    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xrrf9_630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8/node-ca/0.log"
Oct 30 15:38:23.992291 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:23.992248    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k42cc_20a5f545-8ad7-45e5-ae97-1abfeb8d1080/serve-healthcheck-canary/0.log"
Oct 30 15:38:24.782246 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:24.782213    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-trlcx"
Oct 30 15:38:27.604933 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:27.604902    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:27.816654 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:27.816629    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sg7k4"
Oct 30 15:38:28.350762 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.350728    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"]
Oct 30 15:38:28.353436 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.353419    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.369343 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.369321    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 15:38:28.369343 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.369324    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 15:38:28.369505 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.369385    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jhfll\""
Oct 30 15:38:28.381433 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.381408    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 15:38:28.381638 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.381624    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 15:38:28.381831 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.381816    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 15:38:28.394267 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.394243    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"]
Oct 30 15:38:28.456849 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.456810    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.457037 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.456859    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.457037 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.456880    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b254z\" (UniqueName: \"kubernetes.io/projected/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-kube-api-access-b254z\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.457113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.457033    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.557462 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.557419    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.557650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.557478    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.557650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.557511    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.557650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.557528    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b254z\" (UniqueName: \"kubernetes.io/projected/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-kube-api-access-b254z\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.558332 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.558309    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.559987 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.559955    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.559987 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.559973    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.581018 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.580982    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b254z\" (UniqueName: \"kubernetes.io/projected/f80c9a38-1db0-4a00-bd7f-ac7298f2119c-kube-api-access-b254z\") pod \"prometheus-operator-9cd89b74c-4rvjk\" (UID: \"f80c9a38-1db0-4a00-bd7f-ac7298f2119c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.664695 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.664663    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"
Oct 30 15:38:28.975149 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:28.975122    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk"]
Oct 30 15:38:28.976742 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:28.976708    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf80c9a38_1db0_4a00_bd7f_ac7298f2119c.slice/crio-9d7fc577e0581437d43eb86dd9d5124e56932c156a579fa5aa0aef07a8290492 WatchSource:0}: Error finding container 9d7fc577e0581437d43eb86dd9d5124e56932c156a579fa5aa0aef07a8290492: Status 404 returned error can't find the container with id 9d7fc577e0581437d43eb86dd9d5124e56932c156a579fa5aa0aef07a8290492
Oct 30 15:38:29.805715 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:29.805679    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk" event={"ID":"f80c9a38-1db0-4a00-bd7f-ac7298f2119c","Type":"ContainerStarted","Data":"9d7fc577e0581437d43eb86dd9d5124e56932c156a579fa5aa0aef07a8290492"}
Oct 30 15:38:30.810260 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:30.810180    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk" event={"ID":"f80c9a38-1db0-4a00-bd7f-ac7298f2119c","Type":"ContainerStarted","Data":"70aa66c6e59ec1a0ea1b7a77e864b8089104e8923ea52ae69c29b5495c6d378e"}
Oct 30 15:38:30.810260 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:30.810231    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk" event={"ID":"f80c9a38-1db0-4a00-bd7f-ac7298f2119c","Type":"ContainerStarted","Data":"60f0ceb1e47ce6c064d7737133108a6ec79f60833179060a5584afb1aac92f31"}
Oct 30 15:38:30.851732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:30.851669    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-4rvjk" podStartSLOduration=1.415970738 podStartE2EDuration="2.851650495s" totalImagesPullingTime="1.435679757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:38:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:30.851074898 +0000 UTC m=+20.286679246" watchObservedRunningTime="2026-10-30 15:38:30.851650495 +0000 UTC m=+20.287254845"
Oct 30 15:38:33.055153 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.055118    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4tlkp"]
Oct 30 15:38:33.057232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.057216    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.113201 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.113171    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 15:38:33.113382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.113207    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 15:38:33.113382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.113265    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bdgdg\""
Oct 30 15:38:33.113382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.113286    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 15:38:33.194860 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.194819    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6618867-c4d0-40c1-9b00-25f1cf5475c4-sys\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.194860 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.194859    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-textfile\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.195108 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.194881    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6618867-c4d0-40c1-9b00-25f1cf5475c4-root\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.195108 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.194897    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6618867-c4d0-40c1-9b00-25f1cf5475c4-metrics-client-ca\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.195108 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.195019    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-tls\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.195108 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.195065    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmrz8\" (UniqueName: \"kubernetes.io/projected/b6618867-c4d0-40c1-9b00-25f1cf5475c4-kube-api-access-mmrz8\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.195298 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.195127    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.195298 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.195161    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-wtmp\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.195298 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.195184    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.295744 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295704    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.295744 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295743    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-wtmp\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295760    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295823    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6618867-c4d0-40c1-9b00-25f1cf5475c4-sys\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295846    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-textfile\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295873    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6618867-c4d0-40c1-9b00-25f1cf5475c4-root\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295896    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6618867-c4d0-40c1-9b00-25f1cf5475c4-metrics-client-ca\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295920    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-tls\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295927    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-wtmp\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295932    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6618867-c4d0-40c1-9b00-25f1cf5475c4-sys\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295943    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmrz8\" (UniqueName: \"kubernetes.io/projected/b6618867-c4d0-40c1-9b00-25f1cf5475c4-kube-api-access-mmrz8\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.295962    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6618867-c4d0-40c1-9b00-25f1cf5475c4-root\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.296314    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-textfile\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.296483    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.296570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.296486    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6618867-c4d0-40c1-9b00-25f1cf5475c4-metrics-client-ca\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.298147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.298128    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-tls\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.298233 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.298210    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6618867-c4d0-40c1-9b00-25f1cf5475c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.320934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.320877    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmrz8\" (UniqueName: \"kubernetes.io/projected/b6618867-c4d0-40c1-9b00-25f1cf5475c4-kube-api-access-mmrz8\") pod \"node-exporter-4tlkp\" (UID: \"b6618867-c4d0-40c1-9b00-25f1cf5475c4\") " pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.369256 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.369225    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4tlkp"
Oct 30 15:38:33.378900 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:33.378855    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6618867_c4d0_40c1_9b00_25f1cf5475c4.slice/crio-216eac8fa829902f41d2eab3fcd477e36d198074a2fe9f6fbe7af94e8d2a6200 WatchSource:0}: Error finding container 216eac8fa829902f41d2eab3fcd477e36d198074a2fe9f6fbe7af94e8d2a6200: Status 404 returned error can't find the container with id 216eac8fa829902f41d2eab3fcd477e36d198074a2fe9f6fbe7af94e8d2a6200
Oct 30 15:38:33.821258 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:33.821224    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4tlkp" event={"ID":"b6618867-c4d0-40c1-9b00-25f1cf5475c4","Type":"ContainerStarted","Data":"216eac8fa829902f41d2eab3fcd477e36d198074a2fe9f6fbe7af94e8d2a6200"}
Oct 30 15:38:34.826309 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:34.826273    4585 generic.go:356] "Generic (PLEG): container finished" podID="b6618867-c4d0-40c1-9b00-25f1cf5475c4" containerID="904326e5a96254f95716bb7b81649adfd76046bf8173e9ea7e481e58ff52bb92" exitCode=0
Oct 30 15:38:34.826716 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:34.826341    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4tlkp" event={"ID":"b6618867-c4d0-40c1-9b00-25f1cf5475c4","Type":"ContainerDied","Data":"904326e5a96254f95716bb7b81649adfd76046bf8173e9ea7e481e58ff52bb92"}
Oct 30 15:38:35.831776 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:35.831742    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4tlkp" event={"ID":"b6618867-c4d0-40c1-9b00-25f1cf5475c4","Type":"ContainerStarted","Data":"ea192ad2514d725569c79580782d520fa75e67e3c2ab6f743bc84d5e7005b811"}
Oct 30 15:38:35.832231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:35.831796    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4tlkp" event={"ID":"b6618867-c4d0-40c1-9b00-25f1cf5475c4","Type":"ContainerStarted","Data":"d2975744b95d5f212df9fc50bde647647f3afbd954b06a8ef261a770a28287f2"}
Oct 30 15:38:35.899055 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:35.899010    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4tlkp" podStartSLOduration=1.918897399 podStartE2EDuration="2.898995376s" totalImagesPullingTime="908.467912ms" totalInitContainerRuntime="71.630065ms" isStatefulPod=true podCreationTimestamp="2026-10-30 15:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:35.886682255 +0000 UTC m=+25.322286602" watchObservedRunningTime="2026-10-30 15:38:35.898995376 +0000 UTC m=+25.334599757"
Oct 30 15:38:37.609417 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.609388    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d489c8c55-5ptpf"
Oct 30 15:38:37.638464 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.638434    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-56849f5b7c-kj7wx"]
Oct 30 15:38:37.640984 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.640968    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.660996 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.660958    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56849f5b7c-kj7wx"]
Oct 30 15:38:37.669600 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.669556    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 15:38:37.669980 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.669949    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 15:38:37.712617 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.712591    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 15:38:37.712783 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.712698    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 15:38:37.712842 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.712827    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-93shrb6upmiie\""
Oct 30 15:38:37.712886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.712866    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zqfwz\""
Oct 30 15:38:37.726533 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.726502    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e88608ac-eae3-4fef-8cc0-8133c087bf38-audit-log\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.726670 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.726552    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e88608ac-eae3-4fef-8cc0-8133c087bf38-client-ca-bundle\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.726670 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.726628    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e88608ac-eae3-4fef-8cc0-8133c087bf38-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.726746 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.726694    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e88608ac-eae3-4fef-8cc0-8133c087bf38-secret-metrics-server-tls\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.726746 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.726721    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e88608ac-eae3-4fef-8cc0-8133c087bf38-metrics-server-audit-profiles\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.726816 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.726742    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e88608ac-eae3-4fef-8cc0-8133c087bf38-secret-metrics-server-client-certs\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.726816 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.726784    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbxff\" (UniqueName: \"kubernetes.io/projected/e88608ac-eae3-4fef-8cc0-8133c087bf38-kube-api-access-tbxff\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.818126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.818091    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"]
Oct 30 15:38:37.819974 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.819955    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"
Oct 30 15:38:37.827198 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.827167    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e88608ac-eae3-4fef-8cc0-8133c087bf38-audit-log\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.827322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.827225    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e88608ac-eae3-4fef-8cc0-8133c087bf38-client-ca-bundle\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.827322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.827267    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fee57912-cec9-4ffe-8cf6-5a96ae0a2469-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pdlnq\" (UID: \"fee57912-cec9-4ffe-8cf6-5a96ae0a2469\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"
Oct 30 15:38:37.827322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.827309    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e88608ac-eae3-4fef-8cc0-8133c087bf38-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.827485 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.827373    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g49mn\" (UniqueName: \"kubernetes.io/projected/fee57912-cec9-4ffe-8cf6-5a96ae0a2469-kube-api-access-g49mn\") pod \"monitoring-plugin-77f4f76594-pdlnq\" (UID: \"fee57912-cec9-4ffe-8cf6-5a96ae0a2469\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"
Oct 30 15:38:37.827485 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.827412    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e88608ac-eae3-4fef-8cc0-8133c087bf38-secret-metrics-server-tls\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.827485 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.827440    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e88608ac-eae3-4fef-8cc0-8133c087bf38-metrics-server-audit-profiles\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.827485 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.827466    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e88608ac-eae3-4fef-8cc0-8133c087bf38-secret-metrics-server-client-certs\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.827658 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.827489    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbxff\" (UniqueName: \"kubernetes.io/projected/e88608ac-eae3-4fef-8cc0-8133c087bf38-kube-api-access-tbxff\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.827706 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.827687    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e88608ac-eae3-4fef-8cc0-8133c087bf38-audit-log\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.829729 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.829703    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e88608ac-eae3-4fef-8cc0-8133c087bf38-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.830130 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.830104    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e88608ac-eae3-4fef-8cc0-8133c087bf38-metrics-server-audit-profiles\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.830709 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.830690    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e88608ac-eae3-4fef-8cc0-8133c087bf38-secret-metrics-server-tls\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.830788 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.830718    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e88608ac-eae3-4fef-8cc0-8133c087bf38-client-ca-bundle\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.831886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.831859    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e88608ac-eae3-4fef-8cc0-8133c087bf38-secret-metrics-server-client-certs\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.840773 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.840747    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"]
Oct 30 15:38:37.852915 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.852890    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbxff\" (UniqueName: \"kubernetes.io/projected/e88608ac-eae3-4fef-8cc0-8133c087bf38-kube-api-access-tbxff\") pod \"metrics-server-56849f5b7c-kj7wx\" (UID: \"e88608ac-eae3-4fef-8cc0-8133c087bf38\") " pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.920661 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.920634    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 15:38:37.920661 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.920637    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2wxl8\""
Oct 30 15:38:37.928033 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.928003    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fee57912-cec9-4ffe-8cf6-5a96ae0a2469-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pdlnq\" (UID: \"fee57912-cec9-4ffe-8cf6-5a96ae0a2469\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"
Oct 30 15:38:37.928160 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.928047    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g49mn\" (UniqueName: \"kubernetes.io/projected/fee57912-cec9-4ffe-8cf6-5a96ae0a2469-kube-api-access-g49mn\") pod \"monitoring-plugin-77f4f76594-pdlnq\" (UID: \"fee57912-cec9-4ffe-8cf6-5a96ae0a2469\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"
Oct 30 15:38:37.930449 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.930429    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fee57912-cec9-4ffe-8cf6-5a96ae0a2469-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pdlnq\" (UID: \"fee57912-cec9-4ffe-8cf6-5a96ae0a2469\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"
Oct 30 15:38:37.951285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.951258    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:37.956635 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:37.956609    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g49mn\" (UniqueName: \"kubernetes.io/projected/fee57912-cec9-4ffe-8cf6-5a96ae0a2469-kube-api-access-g49mn\") pod \"monitoring-plugin-77f4f76594-pdlnq\" (UID: \"fee57912-cec9-4ffe-8cf6-5a96ae0a2469\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"
Oct 30 15:38:38.131888 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:38.131856    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"
Oct 30 15:38:38.256865 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:38.256597    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56849f5b7c-kj7wx"]
Oct 30 15:38:38.258704 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:38.258674    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode88608ac_eae3_4fef_8cc0_8133c087bf38.slice/crio-35368f07ca819757ba898d05a21efe04319246820e8c2351f6c9be7ff7feb941 WatchSource:0}: Error finding container 35368f07ca819757ba898d05a21efe04319246820e8c2351f6c9be7ff7feb941: Status 404 returned error can't find the container with id 35368f07ca819757ba898d05a21efe04319246820e8c2351f6c9be7ff7feb941
Oct 30 15:38:38.423689 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:38.423662    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"]
Oct 30 15:38:38.425259 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:38.425230    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfee57912_cec9_4ffe_8cf6_5a96ae0a2469.slice/crio-d5ac5a08a551a5507865711ba21fcf8dc2589aca5d6de351696eed587abd3604 WatchSource:0}: Error finding container d5ac5a08a551a5507865711ba21fcf8dc2589aca5d6de351696eed587abd3604: Status 404 returned error can't find the container with id d5ac5a08a551a5507865711ba21fcf8dc2589aca5d6de351696eed587abd3604
Oct 30 15:38:38.845534 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:38.845448    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq" event={"ID":"fee57912-cec9-4ffe-8cf6-5a96ae0a2469","Type":"ContainerStarted","Data":"d5ac5a08a551a5507865711ba21fcf8dc2589aca5d6de351696eed587abd3604"}
Oct 30 15:38:38.846718 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:38.846691    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx" event={"ID":"e88608ac-eae3-4fef-8cc0-8133c087bf38","Type":"ContainerStarted","Data":"35368f07ca819757ba898d05a21efe04319246820e8c2351f6c9be7ff7feb941"}
Oct 30 15:38:39.853056 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:39.853018    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx" event={"ID":"e88608ac-eae3-4fef-8cc0-8133c087bf38","Type":"ContainerStarted","Data":"1214789bfa02c299445ff5a260c1579b7fb99d74874b4355668584c40b1d524a"}
Oct 30 15:38:39.897054 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:39.897005    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx" podStartSLOduration=1.5026369480000001 podStartE2EDuration="2.89699168s" totalImagesPullingTime="1.394354732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:39.896207017 +0000 UTC m=+29.331811364" watchObservedRunningTime="2026-10-30 15:38:39.89699168 +0000 UTC m=+29.332596019"
Oct 30 15:38:40.023758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.023721    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 15:38:40.026397 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.026371    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.042715 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.042681    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 15:38:40.044062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044038    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044070    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044088    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044103    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044396 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044170    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044396 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044215    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044396 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044240    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044396 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044258    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044396 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044304    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044396 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044375    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044666 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044416    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044666 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044478    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044666 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044502    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044666 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044527    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044666 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044555    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044666 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044577    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044943 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044665    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044943 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044693    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwn7k\" (UniqueName: \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-kube-api-access-qwn7k\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044943 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044711    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.044943 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.044726    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.070698 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.068707    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 15:38:40.070698 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.068959    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 15:38:40.070698 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.069092    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 15:38:40.081142 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.081083    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 15:38:40.082566 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.082533    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 15:38:40.126522 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.126492    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 15:38:40.126522 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.126492    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 15:38:40.126522 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.126509    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 15:38:40.126791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.126734    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 15:38:40.126864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.126844    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 15:38:40.126952 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.126935    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 15:38:40.126999 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.126955    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 15:38:40.126999 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.126979    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 15:38:40.126999 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.126992    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-j2p9n\""
Oct 30 15:38:40.127141 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.127018    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-mr361i68mrpn\""
Oct 30 15:38:40.127141 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.127099    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 15:38:40.145866 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.145835    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.146014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.145889    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.146014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.145917    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.146014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.145946    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.146014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.145991    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146027    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146049    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146088    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146119    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwn7k\" (UniqueName: \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-kube-api-access-qwn7k\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146147    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146166    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146211    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146235    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146260    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146287    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146315    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146340    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146389    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146417    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146447    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.146838    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.147233    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.147258    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.147757    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.148981    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.150698    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.151488    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.151705    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.152197    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.152677    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.152844    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config-out\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.153429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.153090    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.154844 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.153887    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.154844 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.154152    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-web-config\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.154844 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.154712    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.154844 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.154726    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.154844 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.154729    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.155250 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.155225    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.156426 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.156406    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.173300 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.173208    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwn7k\" (UniqueName: \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-kube-api-access-qwn7k\") pod \"prometheus-k8s-0\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.348664 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.348374    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:40.689825 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.689796    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 15:38:40.690844 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:38:40.690818    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5761dfd5_60b6_42c4_bc6b_ec9150a9f53a.slice/crio-97a8a1e03806b20492c06a06064b3c7d581383f7dc6f115879dc29a90cafd2c5 WatchSource:0}: Error finding container 97a8a1e03806b20492c06a06064b3c7d581383f7dc6f115879dc29a90cafd2c5: Status 404 returned error can't find the container with id 97a8a1e03806b20492c06a06064b3c7d581383f7dc6f115879dc29a90cafd2c5
Oct 30 15:38:40.865493 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.865452    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerStarted","Data":"97a8a1e03806b20492c06a06064b3c7d581383f7dc6f115879dc29a90cafd2c5"}
Oct 30 15:38:40.867413 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.867378    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq" event={"ID":"fee57912-cec9-4ffe-8cf6-5a96ae0a2469","Type":"ContainerStarted","Data":"ec9fd7ef3d281d3f4fc142b86342dcc1f2856388f327d06045e68d8411db7c21"}
Oct 30 15:38:40.867916 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.867891    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"
Oct 30 15:38:40.872443 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.872419    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq"
Oct 30 15:38:40.908177 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:40.908115    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pdlnq" podStartSLOduration=2.422084182 podStartE2EDuration="3.908098334s" totalImagesPullingTime="1.486014152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:40.907319153 +0000 UTC m=+30.342923517" watchObservedRunningTime="2026-10-30 15:38:40.908098334 +0000 UTC m=+30.343702684"
Oct 30 15:38:41.874633 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:41.874551    4585 generic.go:356] "Generic (PLEG): container finished" podID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerID="ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27" exitCode=0
Oct 30 15:38:41.875033 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:41.874634    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerDied","Data":"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"}
Oct 30 15:38:45.894262 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:45.894230    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerStarted","Data":"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"}
Oct 30 15:38:45.894653 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:45.894306    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerStarted","Data":"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"}
Oct 30 15:38:47.391428 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:47.391398    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ttqnr"
Oct 30 15:38:47.904271 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:47.904193    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerStarted","Data":"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"}
Oct 30 15:38:47.904271 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:47.904257    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerStarted","Data":"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"}
Oct 30 15:38:47.904473 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:47.904281    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerStarted","Data":"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"}
Oct 30 15:38:47.904473 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:47.904304    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerStarted","Data":"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"}
Oct 30 15:38:47.959054 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:47.959023    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:38:47.979661 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:47.979611    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.661284823 podStartE2EDuration="8.979596051s" totalImagesPullingTime="6.30221438s" totalInitContainerRuntime="16.096848ms" isStatefulPod=false podCreationTimestamp="2026-10-30 15:38:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:38:47.978002508 +0000 UTC m=+37.413606837" watchObservedRunningTime="2026-10-30 15:38:47.979596051 +0000 UTC m=+37.415200442"
Oct 30 15:38:50.349630 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:50.349595    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:51.038476 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.038439    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-2zkc5-w5tp6"]
Oct 30 15:38:51.038741 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.038719    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-2zkc5-w5tp6" podUID="e275df6d-03bb-4d35-9b50-f8b5ea98e226" containerName="insights-gathering" containerID="cri-o://65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5" gracePeriod=30
Oct 30 15:38:51.042395 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.042367    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-rssdr-2wg26"]
Oct 30 15:38:51.042566 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.042534    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-rssdr-2wg26" podUID="18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" containerName="insights-gathering" containerID="cri-o://ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844" gracePeriod=30
Oct 30 15:38:51.299894 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.299859    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:51.302583 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.302561    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:51.349116 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349079    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-archives-path\") pod \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") "
Oct 30 15:38:51.349116 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349122    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-trusted-ca-bundle\") pod \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") "
Oct 30 15:38:51.349368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349173    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-kube-api-access-mcrnd\" (UniqueName: \"kubernetes.io/projected/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-kube-api-access-mcrnd\") pod \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") "
Oct 30 15:38:51.349368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349202    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-service-ca-bundle\") pod \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") "
Oct 30 15:38:51.349368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349246    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e275df6d-03bb-4d35-9b50-f8b5ea98e226-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e275df6d-03bb-4d35-9b50-f8b5ea98e226-archives-path\") pod \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") "
Oct 30 15:38:51.349368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349278    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e275df6d-03bb-4d35-9b50-f8b5ea98e226-kube-api-access-vwg82\" (UniqueName: \"kubernetes.io/projected/e275df6d-03bb-4d35-9b50-f8b5ea98e226-kube-api-access-vwg82\") pod \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\" (UID: \"e275df6d-03bb-4d35-9b50-f8b5ea98e226\") "
Oct 30 15:38:51.349368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349314    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-trusted-ca-bundle\") pod \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") "
Oct 30 15:38:51.349610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349421    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-service-ca-bundle\") pod \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\" (UID: \"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3\") "
Oct 30 15:38:51.349610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349511    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-archives-path" pod "18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" (UID: "18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:38:51.350090 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349611    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-trusted-ca-bundle" pod "e275df6d-03bb-4d35-9b50-f8b5ea98e226" (UID: "e275df6d-03bb-4d35-9b50-f8b5ea98e226"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:38:51.350090 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349625    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-service-ca-bundle" pod "e275df6d-03bb-4d35-9b50-f8b5ea98e226" (UID: "e275df6d-03bb-4d35-9b50-f8b5ea98e226"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:38:51.350090 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349845    4585 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-archives-path\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:38:51.350090 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349866    4585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-trusted-ca-bundle\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:38:51.350090 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349879    4585 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e275df6d-03bb-4d35-9b50-f8b5ea98e226-service-ca-bundle\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:38:51.350090 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349904    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-service-ca-bundle" pod "18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" (UID: "18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:38:51.350090 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.349923    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e275df6d-03bb-4d35-9b50-f8b5ea98e226-archives-path" pod "e275df6d-03bb-4d35-9b50-f8b5ea98e226" (UID: "e275df6d-03bb-4d35-9b50-f8b5ea98e226"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:38:51.350090 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.350087    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-trusted-ca-bundle" pod "18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" (UID: "18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:38:51.351613 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.351587    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-kube-api-access-mcrnd" pod "18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" (UID: "18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3"). InnerVolumeSpecName "kube-api-access-mcrnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:38:51.351728 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.351708    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e275df6d-03bb-4d35-9b50-f8b5ea98e226-kube-api-access-vwg82" pod "e275df6d-03bb-4d35-9b50-f8b5ea98e226" (UID: "e275df6d-03bb-4d35-9b50-f8b5ea98e226"). InnerVolumeSpecName "kube-api-access-vwg82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:38:51.451217 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.451185    4585 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-service-ca-bundle\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:38:51.451217 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.451216    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcrnd\" (UniqueName: \"kubernetes.io/projected/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-kube-api-access-mcrnd\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:38:51.451430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.451229    4585 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/e275df6d-03bb-4d35-9b50-f8b5ea98e226-archives-path\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:38:51.451430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.451241    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwg82\" (UniqueName: \"kubernetes.io/projected/e275df6d-03bb-4d35-9b50-f8b5ea98e226-kube-api-access-vwg82\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:38:51.451430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.451255    4585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3-trusted-ca-bundle\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:38:51.919140 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.919107    4585 generic.go:356] "Generic (PLEG): container finished" podID="e275df6d-03bb-4d35-9b50-f8b5ea98e226" containerID="65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5" exitCode=2
Oct 30 15:38:51.919364 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.919167    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-2zkc5-w5tp6"
Oct 30 15:38:51.919364 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.919197    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-2zkc5-w5tp6" event={"ID":"e275df6d-03bb-4d35-9b50-f8b5ea98e226","Type":"ContainerDied","Data":"65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5"}
Oct 30 15:38:51.919364 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.919291    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-2zkc5-w5tp6" event={"ID":"e275df6d-03bb-4d35-9b50-f8b5ea98e226","Type":"ContainerDied","Data":"279c51ea331e8569b31b535e1ac8d6a901efefe06e58e868d0ba0d88f5e7c100"}
Oct 30 15:38:51.919364 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.919319    4585 scope.go:122] "RemoveContainer" containerID="65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5"
Oct 30 15:38:51.920586 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.920563    4585 generic.go:356] "Generic (PLEG): container finished" podID="18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" containerID="ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844" exitCode=2
Oct 30 15:38:51.920692 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.920604    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rssdr-2wg26" event={"ID":"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3","Type":"ContainerDied","Data":"ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844"}
Oct 30 15:38:51.920692 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.920641    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rssdr-2wg26" event={"ID":"18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3","Type":"ContainerDied","Data":"9b9096edeb1ab5811232cdeaaf985cb99938bbbdec270bd2c89a7962a39cc45a"}
Oct 30 15:38:51.920692 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.920646    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rssdr-2wg26"
Oct 30 15:38:51.928217 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.928012    4585 scope.go:122] "RemoveContainer" containerID="65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5"
Oct 30 15:38:51.928378 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:51.928339    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5\": container with ID starting with 65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5 not found: ID does not exist" containerID="65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5"
Oct 30 15:38:51.928484 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.928388    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5"} err="failed to get container status \"65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5\": rpc error: code = NotFound desc = could not find container \"65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5\": container with ID starting with 65dce9ffaa2971de0a2ed0ec873da258e99ecdb7aab6d6f8e29f7b395e819ff5 not found: ID does not exist"
Oct 30 15:38:51.928484 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.928430    4585 scope.go:122] "RemoveContainer" containerID="ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844"
Oct 30 15:38:51.935750 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.935734    4585 scope.go:122] "RemoveContainer" containerID="ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844"
Oct 30 15:38:51.935993 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:38:51.935972    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844\": container with ID starting with ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844 not found: ID does not exist" containerID="ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844"
Oct 30 15:38:51.936040 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.936001    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844"} err="failed to get container status \"ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844\": rpc error: code = NotFound desc = could not find container \"ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844\": container with ID starting with ff0ab9387b6b886056e2a65100527112c859d00471978b4f61c84725427eb844 not found: ID does not exist"
Oct 30 15:38:51.970175 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.970142    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-rssdr-2wg26"]
Oct 30 15:38:51.982282 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:51.982246    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-rssdr-2wg26"]
Oct 30 15:38:52.005931 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:52.005897    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-2zkc5-w5tp6"]
Oct 30 15:38:52.010147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:52.010122    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-2zkc5-w5tp6"]
Oct 30 15:38:52.699380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:52.699331    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" path="/var/lib/kubelet/pods/18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3/volumes"
Oct 30 15:38:52.699760 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:52.699731    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e275df6d-03bb-4d35-9b50-f8b5ea98e226" path="/var/lib/kubelet/pods/e275df6d-03bb-4d35-9b50-f8b5ea98e226/volumes"
Oct 30 15:38:56.875714 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:56.875623    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:56.894154 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:56.894131    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:56.951532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:56.951503    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:38:57.951597 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:57.951568    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:38:57.952105 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:38:57.951659    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:39:17.957879 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:17.957833    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:39:17.960483 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:39:17.960450    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:39:17.961926 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:17.961904    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-56849f5b7c-kj7wx"
Oct 30 15:39:28.551830 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.551791    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 15:39:28.552460 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.552174    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="prometheus" containerID="cri-o://e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea" gracePeriod=600
Oct 30 15:39:28.552460 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.552196    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f" gracePeriod=600
Oct 30 15:39:28.552460 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.552212    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="thanos-sidecar" containerID="cri-o://b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3" gracePeriod=600
Oct 30 15:39:28.552460 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.552274    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="config-reloader" containerID="cri-o://dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5" gracePeriod=600
Oct 30 15:39:28.552460 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.552229    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy-web" containerID="cri-o://dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237" gracePeriod=600
Oct 30 15:39:28.552460 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.552247    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy" containerID="cri-o://80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc" gracePeriod=600
Oct 30 15:39:28.813160 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.813135    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:28.953329 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953292    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-grpc-tls\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.953530 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953404    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-db\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.953530 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953446    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-2\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.953530 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953476    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config-out\" (UniqueName: \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config-out\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.953530 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953500    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-metrics-client-certs\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.953530 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953528    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-1\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953590    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-kube-api-access-qwn7k\" (UniqueName: \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-kube-api-access-qwn7k\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953636    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953671    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-kubelet-serving-ca-bundle\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953701    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-thanos-prometheus-http-client-file\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953739    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-trusted-ca-bundle\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953764    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-metrics-client-ca\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953803    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-kube-rbac-proxy\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953835    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953874    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-tls\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953914    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-0\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953951    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.953991    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-serving-certs-ca-bundle\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.954035    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-web-config\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-web-config\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.954065    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-tls-assets\" (UniqueName: \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-tls-assets\") pod \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\" (UID: \"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a\") "
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.954406    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-2" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:39:28.956230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.956029    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-grpc-tls" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:39:28.957182 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.957149    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-1" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:39:28.957383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.957249    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-db" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:39:28.960610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.959770    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-metrics-client-certs" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:39:28.960610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.959860    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-kube-rbac-proxy" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:39:28.960610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.959897    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-kube-rbac-proxy-web" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:39:28.960610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.960372    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-kube-api-access-qwn7k" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "kube-api-access-qwn7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:39:28.960610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.960497    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-metrics-client-ca" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:39:28.960610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.960487    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-kubelet-serving-ca-bundle" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:39:28.960966 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.960912    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-tls" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:39:28.960966 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.960954    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-tls-assets" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:39:28.964456 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.961259    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-serving-certs-ca-bundle" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:39:28.964456 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.961431    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-trusted-ca-bundle" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:39:28.964456 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.962083    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-thanos-prometheus-http-client-file" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:39:28.964456 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.962989    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-0" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:39:28.967382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.967262    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:39:28.967382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.967319    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config-out" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:39:28.968580 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.968552    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-thanos-sidecar-tls" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:39:28.978177 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:28.978154    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-web-config" pod "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" (UID: "5761dfd5-60b6-42c4-bc6b-ec9150a9f53a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:39:29.047231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047196    4585 generic.go:356] "Generic (PLEG): container finished" podID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerID="d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f" exitCode=0
Oct 30 15:39:29.047231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047222    4585 generic.go:356] "Generic (PLEG): container finished" podID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerID="80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc" exitCode=0
Oct 30 15:39:29.047231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047229    4585 generic.go:356] "Generic (PLEG): container finished" podID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerID="dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237" exitCode=0
Oct 30 15:39:29.047231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047237    4585 generic.go:356] "Generic (PLEG): container finished" podID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerID="b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3" exitCode=0
Oct 30 15:39:29.047231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047242    4585 generic.go:356] "Generic (PLEG): container finished" podID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerID="dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5" exitCode=0
Oct 30 15:39:29.047646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047250    4585 generic.go:356] "Generic (PLEG): container finished" podID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerID="e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea" exitCode=0
Oct 30 15:39:29.047646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047295    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerDied","Data":"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"}
Oct 30 15:39:29.047646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047309    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.047646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047458    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerDied","Data":"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"}
Oct 30 15:39:29.047646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047510    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerDied","Data":"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"}
Oct 30 15:39:29.047646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047518    4585 scope.go:122] "RemoveContainer" containerID="d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"
Oct 30 15:39:29.047646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047553    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerDied","Data":"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"}
Oct 30 15:39:29.047646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047596    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerDied","Data":"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"}
Oct 30 15:39:29.047646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047637    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerDied","Data":"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"}
Oct 30 15:39:29.048082 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.047678    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5761dfd5-60b6-42c4-bc6b-ec9150a9f53a","Type":"ContainerDied","Data":"97a8a1e03806b20492c06a06064b3c7d581383f7dc6f115879dc29a90cafd2c5"}
Oct 30 15:39:29.054845 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054790    4585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.054845 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054811    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwn7k\" (UniqueName: \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-kube-api-access-qwn7k\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.054845 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054827    4585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.054845 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054842    4585 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054856    4585 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054869    4585 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054881    4585 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-metrics-client-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054893    4585 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-kube-rbac-proxy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054905    4585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054918    4585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054930    4585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054942    4585 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054954    4585 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.054992    4585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-web-config\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.055008    4585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-tls-assets\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.055021    4585 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-grpc-tls\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.055033    4585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-db\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.055047    4585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.055061    4585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-config-out\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.055074    4585 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a-secret-metrics-client-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:39:29.055837 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.055811    4585 scope.go:122] "RemoveContainer" containerID="80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"
Oct 30 15:39:29.064480 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.063857    4585 scope.go:122] "RemoveContainer" containerID="dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"
Oct 30 15:39:29.070967 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.070950    4585 scope.go:122] "RemoveContainer" containerID="b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"
Oct 30 15:39:29.077276 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.077257    4585 scope.go:122] "RemoveContainer" containerID="dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"
Oct 30 15:39:29.083999 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.083976    4585 scope.go:122] "RemoveContainer" containerID="e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"
Oct 30 15:39:29.091539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.091524    4585 scope.go:122] "RemoveContainer" containerID="ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"
Oct 30 15:39:29.097638 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.097622    4585 scope.go:122] "RemoveContainer" containerID="d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"
Oct 30 15:39:29.097900 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:39:29.097881    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": container with ID starting with d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f not found: ID does not exist" containerID="d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"
Oct 30 15:39:29.097940 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.097910    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"} err="failed to get container status \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": rpc error: code = NotFound desc = could not find container \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": container with ID starting with d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f not found: ID does not exist"
Oct 30 15:39:29.097940 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.097927    4585 scope.go:122] "RemoveContainer" containerID="80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"
Oct 30 15:39:29.098180 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:39:29.098162    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": container with ID starting with 80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc not found: ID does not exist" containerID="80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"
Oct 30 15:39:29.098236 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.098187    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"} err="failed to get container status \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": rpc error: code = NotFound desc = could not find container \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": container with ID starting with 80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc not found: ID does not exist"
Oct 30 15:39:29.098236 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.098204    4585 scope.go:122] "RemoveContainer" containerID="dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"
Oct 30 15:39:29.098459 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:39:29.098442    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": container with ID starting with dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237 not found: ID does not exist" containerID="dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"
Oct 30 15:39:29.098506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.098462    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"} err="failed to get container status \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": rpc error: code = NotFound desc = could not find container \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": container with ID starting with dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237 not found: ID does not exist"
Oct 30 15:39:29.098506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.098474    4585 scope.go:122] "RemoveContainer" containerID="b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"
Oct 30 15:39:29.098711 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:39:29.098693    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": container with ID starting with b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3 not found: ID does not exist" containerID="b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"
Oct 30 15:39:29.098755 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.098716    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"} err="failed to get container status \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": rpc error: code = NotFound desc = could not find container \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": container with ID starting with b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3 not found: ID does not exist"
Oct 30 15:39:29.098755 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.098731    4585 scope.go:122] "RemoveContainer" containerID="dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"
Oct 30 15:39:29.098947 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:39:29.098930    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": container with ID starting with dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5 not found: ID does not exist" containerID="dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"
Oct 30 15:39:29.098987 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.098952    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"} err="failed to get container status \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": rpc error: code = NotFound desc = could not find container \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": container with ID starting with dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5 not found: ID does not exist"
Oct 30 15:39:29.098987 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.098965    4585 scope.go:122] "RemoveContainer" containerID="e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"
Oct 30 15:39:29.099165 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:39:29.099149    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": container with ID starting with e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea not found: ID does not exist" containerID="e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"
Oct 30 15:39:29.099211 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.099170    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"} err="failed to get container status \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": rpc error: code = NotFound desc = could not find container \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": container with ID starting with e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea not found: ID does not exist"
Oct 30 15:39:29.099211 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.099189    4585 scope.go:122] "RemoveContainer" containerID="ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"
Oct 30 15:39:29.099461 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:39:29.099441    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": container with ID starting with ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27 not found: ID does not exist" containerID="ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"
Oct 30 15:39:29.099538 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.099467    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"} err="failed to get container status \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": rpc error: code = NotFound desc = could not find container \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": container with ID starting with ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27 not found: ID does not exist"
Oct 30 15:39:29.099538 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.099487    4585 scope.go:122] "RemoveContainer" containerID="d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"
Oct 30 15:39:29.099750 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.099733    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"} err="failed to get container status \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": rpc error: code = NotFound desc = could not find container \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": container with ID starting with d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f not found: ID does not exist"
Oct 30 15:39:29.099794 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.099751    4585 scope.go:122] "RemoveContainer" containerID="80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"
Oct 30 15:39:29.099987 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.099970    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"} err="failed to get container status \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": rpc error: code = NotFound desc = could not find container \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": container with ID starting with 80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc not found: ID does not exist"
Oct 30 15:39:29.100037 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.099988    4585 scope.go:122] "RemoveContainer" containerID="dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"
Oct 30 15:39:29.100243 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.100225    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"} err="failed to get container status \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": rpc error: code = NotFound desc = could not find container \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": container with ID starting with dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237 not found: ID does not exist"
Oct 30 15:39:29.100243 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.100240    4585 scope.go:122] "RemoveContainer" containerID="b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"
Oct 30 15:39:29.100521 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.100502    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"} err="failed to get container status \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": rpc error: code = NotFound desc = could not find container \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": container with ID starting with b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3 not found: ID does not exist"
Oct 30 15:39:29.100521 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.100519    4585 scope.go:122] "RemoveContainer" containerID="dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"
Oct 30 15:39:29.100768 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.100748    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"} err="failed to get container status \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": rpc error: code = NotFound desc = could not find container \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": container with ID starting with dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5 not found: ID does not exist"
Oct 30 15:39:29.100768 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.100766    4585 scope.go:122] "RemoveContainer" containerID="e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"
Oct 30 15:39:29.101179 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.101019    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"} err="failed to get container status \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": rpc error: code = NotFound desc = could not find container \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": container with ID starting with e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea not found: ID does not exist"
Oct 30 15:39:29.101179 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.101043    4585 scope.go:122] "RemoveContainer" containerID="ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"
Oct 30 15:39:29.101290 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.101271    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"} err="failed to get container status \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": rpc error: code = NotFound desc = could not find container \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": container with ID starting with ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27 not found: ID does not exist"
Oct 30 15:39:29.101290 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.101287    4585 scope.go:122] "RemoveContainer" containerID="d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"
Oct 30 15:39:29.101521 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.101505    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"} err="failed to get container status \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": rpc error: code = NotFound desc = could not find container \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": container with ID starting with d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f not found: ID does not exist"
Oct 30 15:39:29.101585 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.101522    4585 scope.go:122] "RemoveContainer" containerID="80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"
Oct 30 15:39:29.101807 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.101751    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"} err="failed to get container status \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": rpc error: code = NotFound desc = could not find container \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": container with ID starting with 80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc not found: ID does not exist"
Oct 30 15:39:29.101858 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.101809    4585 scope.go:122] "RemoveContainer" containerID="dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"
Oct 30 15:39:29.102062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.102028    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"} err="failed to get container status \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": rpc error: code = NotFound desc = could not find container \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": container with ID starting with dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237 not found: ID does not exist"
Oct 30 15:39:29.102062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.102050    4585 scope.go:122] "RemoveContainer" containerID="b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"
Oct 30 15:39:29.102355 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.102326    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"} err="failed to get container status \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": rpc error: code = NotFound desc = could not find container \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": container with ID starting with b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3 not found: ID does not exist"
Oct 30 15:39:29.102468 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.102363    4585 scope.go:122] "RemoveContainer" containerID="dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"
Oct 30 15:39:29.102614 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.102594    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"} err="failed to get container status \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": rpc error: code = NotFound desc = could not find container \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": container with ID starting with dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5 not found: ID does not exist"
Oct 30 15:39:29.102689 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.102616    4585 scope.go:122] "RemoveContainer" containerID="e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"
Oct 30 15:39:29.102890 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.102872    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"} err="failed to get container status \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": rpc error: code = NotFound desc = could not find container \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": container with ID starting with e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea not found: ID does not exist"
Oct 30 15:39:29.102960 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.102891    4585 scope.go:122] "RemoveContainer" containerID="ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"
Oct 30 15:39:29.103117 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.103101    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"} err="failed to get container status \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": rpc error: code = NotFound desc = could not find container \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": container with ID starting with ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27 not found: ID does not exist"
Oct 30 15:39:29.103184 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.103118    4585 scope.go:122] "RemoveContainer" containerID="d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"
Oct 30 15:39:29.103399 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.103375    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"} err="failed to get container status \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": rpc error: code = NotFound desc = could not find container \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": container with ID starting with d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f not found: ID does not exist"
Oct 30 15:39:29.103451 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.103399    4585 scope.go:122] "RemoveContainer" containerID="80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"
Oct 30 15:39:29.103451 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.103407    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 15:39:29.103650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.103632    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"} err="failed to get container status \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": rpc error: code = NotFound desc = could not find container \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": container with ID starting with 80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc not found: ID does not exist"
Oct 30 15:39:29.103695 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.103651    4585 scope.go:122] "RemoveContainer" containerID="dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"
Oct 30 15:39:29.103872 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.103854    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"} err="failed to get container status \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": rpc error: code = NotFound desc = could not find container \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": container with ID starting with dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237 not found: ID does not exist"
Oct 30 15:39:29.103935 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.103873    4585 scope.go:122] "RemoveContainer" containerID="b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"
Oct 30 15:39:29.104114 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.104095    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"} err="failed to get container status \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": rpc error: code = NotFound desc = could not find container \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": container with ID starting with b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3 not found: ID does not exist"
Oct 30 15:39:29.104158 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.104115    4585 scope.go:122] "RemoveContainer" containerID="dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"
Oct 30 15:39:29.104379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.104360    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"} err="failed to get container status \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": rpc error: code = NotFound desc = could not find container \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": container with ID starting with dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5 not found: ID does not exist"
Oct 30 15:39:29.104465 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.104380    4585 scope.go:122] "RemoveContainer" containerID="e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"
Oct 30 15:39:29.104593 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.104576    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"} err="failed to get container status \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": rpc error: code = NotFound desc = could not find container \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": container with ID starting with e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea not found: ID does not exist"
Oct 30 15:39:29.104659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.104593    4585 scope.go:122] "RemoveContainer" containerID="ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"
Oct 30 15:39:29.104791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.104775    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"} err="failed to get container status \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": rpc error: code = NotFound desc = could not find container \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": container with ID starting with ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27 not found: ID does not exist"
Oct 30 15:39:29.104855 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.104792    4585 scope.go:122] "RemoveContainer" containerID="d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"
Oct 30 15:39:29.104980 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.104964    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"} err="failed to get container status \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": rpc error: code = NotFound desc = could not find container \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": container with ID starting with d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f not found: ID does not exist"
Oct 30 15:39:29.105045 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.104981    4585 scope.go:122] "RemoveContainer" containerID="80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"
Oct 30 15:39:29.105185 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.105169    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"} err="failed to get container status \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": rpc error: code = NotFound desc = could not find container \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": container with ID starting with 80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc not found: ID does not exist"
Oct 30 15:39:29.105253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.105185    4585 scope.go:122] "RemoveContainer" containerID="dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"
Oct 30 15:39:29.105463 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.105445    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"} err="failed to get container status \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": rpc error: code = NotFound desc = could not find container \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": container with ID starting with dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237 not found: ID does not exist"
Oct 30 15:39:29.105551 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.105463    4585 scope.go:122] "RemoveContainer" containerID="b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"
Oct 30 15:39:29.105706 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.105689    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"} err="failed to get container status \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": rpc error: code = NotFound desc = could not find container \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": container with ID starting with b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3 not found: ID does not exist"
Oct 30 15:39:29.105891 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.105706    4585 scope.go:122] "RemoveContainer" containerID="dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"
Oct 30 15:39:29.106068 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.106043    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"} err="failed to get container status \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": rpc error: code = NotFound desc = could not find container \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": container with ID starting with dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5 not found: ID does not exist"
Oct 30 15:39:29.106135 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.106069    4585 scope.go:122] "RemoveContainer" containerID="e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"
Oct 30 15:39:29.106380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.106330    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"} err="failed to get container status \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": rpc error: code = NotFound desc = could not find container \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": container with ID starting with e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea not found: ID does not exist"
Oct 30 15:39:29.106468 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.106412    4585 scope.go:122] "RemoveContainer" containerID="ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"
Oct 30 15:39:29.106682 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.106650    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"} err="failed to get container status \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": rpc error: code = NotFound desc = could not find container \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": container with ID starting with ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27 not found: ID does not exist"
Oct 30 15:39:29.106769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.106677    4585 scope.go:122] "RemoveContainer" containerID="d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"
Oct 30 15:39:29.106981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.106959    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f"} err="failed to get container status \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": rpc error: code = NotFound desc = could not find container \"d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f\": container with ID starting with d3fed2b1d06f5b551cd163dccfec3bab204d14c76031edf13e2736a79a13d53f not found: ID does not exist"
Oct 30 15:39:29.106981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.106980    4585 scope.go:122] "RemoveContainer" containerID="80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"
Oct 30 15:39:29.107190 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.107172    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc"} err="failed to get container status \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": rpc error: code = NotFound desc = could not find container \"80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc\": container with ID starting with 80aba47ca8885ee87fd89b951c04c57df5bd8583d7034d4c4d9d93447c531dcc not found: ID does not exist"
Oct 30 15:39:29.107262 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.107196    4585 scope.go:122] "RemoveContainer" containerID="dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"
Oct 30 15:39:29.107423 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.107405    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237"} err="failed to get container status \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": rpc error: code = NotFound desc = could not find container \"dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237\": container with ID starting with dcdc03f7bdc09236ab3b9920f7ed4c6772cfbbce961e77ffef37233bd919f237 not found: ID does not exist"
Oct 30 15:39:29.107423 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.107423    4585 scope.go:122] "RemoveContainer" containerID="b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"
Oct 30 15:39:29.107620 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.107599    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3"} err="failed to get container status \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": rpc error: code = NotFound desc = could not find container \"b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3\": container with ID starting with b32dd34d7bf9a4202ff8a34f10ddded7c673f7d3150f725c87c4fd4d0749b0e3 not found: ID does not exist"
Oct 30 15:39:29.107620 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.107620    4585 scope.go:122] "RemoveContainer" containerID="dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"
Oct 30 15:39:29.107837 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.107818    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5"} err="failed to get container status \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": rpc error: code = NotFound desc = could not find container \"dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5\": container with ID starting with dd36c235d47886067a1793e283dff11d43a6d684a672275a5806889589e832e5 not found: ID does not exist"
Oct 30 15:39:29.107896 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.107837    4585 scope.go:122] "RemoveContainer" containerID="e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"
Oct 30 15:39:29.107955 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.107937    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 15:39:29.108051 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.108035    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea"} err="failed to get container status \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": rpc error: code = NotFound desc = could not find container \"e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea\": container with ID starting with e739c6ed45bb9be8896e158b90ae667e9bdabcdf9581f4460541b08c6ba3d2ea not found: ID does not exist"
Oct 30 15:39:29.108092 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.108051    4585 scope.go:122] "RemoveContainer" containerID="ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"
Oct 30 15:39:29.108282 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.108266    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27"} err="failed to get container status \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": rpc error: code = NotFound desc = could not find container \"ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27\": container with ID starting with ccb88d64321f5d0a2f28710e06faaa8b8e55af4b9c00b1fa12abceb3ade94e27 not found: ID does not exist"
Oct 30 15:39:29.146773 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.146740    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 15:39:29.146985 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.146972    4585 cpu_manager.go:414] "containerMap: removing container" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="prometheus"
Oct 30 15:39:29.147039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.146986    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="prometheus"
Oct 30 15:39:29.147039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.146995    4585 cpu_manager.go:414] "containerMap: removing container" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy-thanos"
Oct 30 15:39:29.147039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147000    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy-thanos"
Oct 30 15:39:29.147039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147007    4585 cpu_manager.go:414] "containerMap: removing container" podUID="18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" containerName="insights-gathering"
Oct 30 15:39:29.147039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147012    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" containerName="insights-gathering"
Oct 30 15:39:29.147039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147024    4585 cpu_manager.go:414] "containerMap: removing container" podUID="e275df6d-03bb-4d35-9b50-f8b5ea98e226" containerName="insights-gathering"
Oct 30 15:39:29.147039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147029    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e275df6d-03bb-4d35-9b50-f8b5ea98e226" containerName="insights-gathering"
Oct 30 15:39:29.147039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147034    4585 cpu_manager.go:414] "containerMap: removing container" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy"
Oct 30 15:39:29.147039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147039    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147047    4585 cpu_manager.go:414] "containerMap: removing container" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="config-reloader"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147051    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="config-reloader"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147057    4585 cpu_manager.go:414] "containerMap: removing container" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="thanos-sidecar"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147061    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="thanos-sidecar"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147067    4585 cpu_manager.go:414] "containerMap: removing container" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy-web"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147071    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy-web"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147083    4585 cpu_manager.go:414] "containerMap: removing container" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="init-config-reloader"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147087    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="init-config-reloader"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147122    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="prometheus"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147130    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy-thanos"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147135    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="18a8a0fc-46c9-44ae-b4cf-76e3a17ed1e3" containerName="insights-gathering"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147142    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="e275df6d-03bb-4d35-9b50-f8b5ea98e226" containerName="insights-gathering"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147147    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147152    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="config-reloader"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147157    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="thanos-sidecar"
Oct 30 15:39:29.147285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.147162    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" containerName="kube-rbac-proxy-web"
Oct 30 15:39:29.151735 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.151715    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.169722 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.169695    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 15:39:29.169862 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.169726    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 15:39:29.169862 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.169811    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 15:39:29.169954 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.169872    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 15:39:29.169954 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.169908    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 15:39:29.170041 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.169954    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 15:39:29.170041 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.170019    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 15:39:29.170462 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.170442    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-mr361i68mrpn\""
Oct 30 15:39:29.170548 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.170452    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 15:39:29.170746 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.170722    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 15:39:29.170810 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.170757    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-j2p9n\""
Oct 30 15:39:29.180655 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.180633    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 15:39:29.183899 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.183874    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 15:39:29.184019 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.184004    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 15:39:29.184237 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.184223    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 15:39:29.196409 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.196390    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 15:39:29.196552 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.196514    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 15:39:29.256592 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256558    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-config-out\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.256749 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256599    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.256749 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256619    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.256749 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256672    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bcsx\" (UniqueName: \"kubernetes.io/projected/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-kube-api-access-9bcsx\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.256749 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256735    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.256892 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256774    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.256892 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256865    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.256952 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256897    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.256952 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256923    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257011 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256956    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257011 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.256990    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257069 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.257029    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257069 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.257060    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257129 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.257102    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257129 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.257118    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.257139    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-config\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.257161    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257252 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.257223    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257284 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.257260    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.257313 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.257282    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-web-config\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358121 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358034    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358121 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358077    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358121 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358096    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358121 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358112    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358131    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358148    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358177    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-config\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358207    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358234    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358261    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358286    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-web-config\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358321    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-config-out\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358374    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.358446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358396    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359037 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358980    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359037 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.358992    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359197 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.359174    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359586 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.359560    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359690 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.359621    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bcsx\" (UniqueName: \"kubernetes.io/projected/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-kube-api-access-9bcsx\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359690 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.359659    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359796 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.359687    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359796 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.359726    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359796 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.359766    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359941 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.359794    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.359941 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.359801    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.360050 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.359991    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.360646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.360619    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.361834 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.361308    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-config-out\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.361834 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.361794    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.362434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.362383    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.362568 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.362446    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.363372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.363048    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.363482 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.363424    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.363482 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.363463    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.363587 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.363482    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-config\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.363587 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.363466    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.363666 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.363583    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.364265 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.364249    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.364678 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.364662    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-web-config\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.383645 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.383626    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bcsx\" (UniqueName: \"kubernetes.io/projected/54566ffd-8215-4bfa-80fb-3eabf5c68ee0-kube-api-access-9bcsx\") pod \"prometheus-k8s-0\" (UID: \"54566ffd-8215-4bfa-80fb-3eabf5c68ee0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.474546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.474514    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:29.808187 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:29.808163    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 15:39:29.809722 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:39:29.809695    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54566ffd_8215_4bfa_80fb_3eabf5c68ee0.slice/crio-d02de6853983d2f9d8ce95c30cbec14140cb5e25e6a76a9bee6d7615b3c5a552 WatchSource:0}: Error finding container d02de6853983d2f9d8ce95c30cbec14140cb5e25e6a76a9bee6d7615b3c5a552: Status 404 returned error can't find the container with id d02de6853983d2f9d8ce95c30cbec14140cb5e25e6a76a9bee6d7615b3c5a552
Oct 30 15:39:30.053157 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:30.053122    4585 generic.go:356] "Generic (PLEG): container finished" podID="54566ffd-8215-4bfa-80fb-3eabf5c68ee0" containerID="b04d3f52b2683fc71a02646d5e8e0aae8c22d0534eb8fd8b95df8d13a4c98151" exitCode=0
Oct 30 15:39:30.053339 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:30.053205    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54566ffd-8215-4bfa-80fb-3eabf5c68ee0","Type":"ContainerDied","Data":"b04d3f52b2683fc71a02646d5e8e0aae8c22d0534eb8fd8b95df8d13a4c98151"}
Oct 30 15:39:30.053339 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:30.053275    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54566ffd-8215-4bfa-80fb-3eabf5c68ee0","Type":"ContainerStarted","Data":"d02de6853983d2f9d8ce95c30cbec14140cb5e25e6a76a9bee6d7615b3c5a552"}
Oct 30 15:39:30.699095 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:30.699059    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5761dfd5-60b6-42c4-bc6b-ec9150a9f53a" path="/var/lib/kubelet/pods/5761dfd5-60b6-42c4-bc6b-ec9150a9f53a/volumes"
Oct 30 15:39:31.059769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:31.059734    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54566ffd-8215-4bfa-80fb-3eabf5c68ee0","Type":"ContainerStarted","Data":"cbd18bca99534f417313d4490172aee556ed4b9edcf802f7c8e9bc7db63f1595"}
Oct 30 15:39:31.060146 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:31.059814    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54566ffd-8215-4bfa-80fb-3eabf5c68ee0","Type":"ContainerStarted","Data":"f6e78087b84e5740e6053f470c3bbeeaa5db343e28be25c591be62a0e973b66b"}
Oct 30 15:39:31.060146 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:31.059856    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54566ffd-8215-4bfa-80fb-3eabf5c68ee0","Type":"ContainerStarted","Data":"c1e11d8b0009100ecfc7e8a806009b0ba60a99023c45abadf8160865494e96e9"}
Oct 30 15:39:31.060146 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:31.059895    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54566ffd-8215-4bfa-80fb-3eabf5c68ee0","Type":"ContainerStarted","Data":"1bdcccd52deb5e5fb60e545bbe0bf88b78e689cb561c146510528f40351a41a9"}
Oct 30 15:39:31.060146 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:31.059930    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54566ffd-8215-4bfa-80fb-3eabf5c68ee0","Type":"ContainerStarted","Data":"0aae356345529f2aa277c54b7ed62d948a8bbb48f6c582af86b3393df6fe86ee"}
Oct 30 15:39:31.060146 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:31.059965    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"54566ffd-8215-4bfa-80fb-3eabf5c68ee0","Type":"ContainerStarted","Data":"2d17c35d7702e3f23e52db644909e6db799b6e1b42bb94dad6b1ba27cb708a70"}
Oct 30 15:39:31.128501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:31.128446    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.112430128 podStartE2EDuration="2.128427121s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.996993ms" isStatefulPod=false podCreationTimestamp="2026-10-30 15:39:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:39:31.127124473 +0000 UTC m=+80.562728834" watchObservedRunningTime="2026-10-30 15:39:31.128427121 +0000 UTC m=+80.564031471"
Oct 30 15:39:34.475436 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:39:34.475387    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:39:47.960420 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:39:47.960383    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:40:17.959799 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:40:17.959766    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:40:29.475738 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:40:29.475645    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:40:29.490919 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:40:29.490891    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:40:30.269554 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:40:30.269523    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 15:40:47.960518 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:40:47.960484    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:41:17.959587 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:41:17.959553    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:41:47.960840 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:41:47.960802    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:42:17.959380 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:42:17.959311    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:42:47.960395 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:42:47.960338    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:43:05.606451 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.606410    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"]
Oct 30 15:43:05.609552 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.609534    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.625273 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.625246    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 15:43:05.625484 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.625301    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 15:43:05.625484 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.625308    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gqp9p\""
Oct 30 15:43:05.637707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.637681    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 15:43:05.637773 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.637686    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 15:43:05.637773 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.637710    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\""
Oct 30 15:43:05.642734 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.642702    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"]
Oct 30 15:43:05.717189 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.717151    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/252b401b-dddf-424b-9ff0-6c4bf5b31748-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.717189 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.717188    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/252b401b-dddf-424b-9ff0-6c4bf5b31748-tmp\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.717424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.717216    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd7bx\" (UniqueName: \"kubernetes.io/projected/252b401b-dddf-424b-9ff0-6c4bf5b31748-kube-api-access-zd7bx\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.717424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.717324    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/252b401b-dddf-424b-9ff0-6c4bf5b31748-metrics-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.717424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.717372    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/252b401b-dddf-424b-9ff0-6c4bf5b31748-manifests\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.717521 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.717478    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/252b401b-dddf-424b-9ff0-6c4bf5b31748-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.818134 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.818074    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/252b401b-dddf-424b-9ff0-6c4bf5b31748-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.818134 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.818138    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/252b401b-dddf-424b-9ff0-6c4bf5b31748-tmp\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.818392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.818184    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd7bx\" (UniqueName: \"kubernetes.io/projected/252b401b-dddf-424b-9ff0-6c4bf5b31748-kube-api-access-zd7bx\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.818392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.818212    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/252b401b-dddf-424b-9ff0-6c4bf5b31748-metrics-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.818392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.818235    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/252b401b-dddf-424b-9ff0-6c4bf5b31748-manifests\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.818392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.818284    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/252b401b-dddf-424b-9ff0-6c4bf5b31748-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.818736 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.818706    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/252b401b-dddf-424b-9ff0-6c4bf5b31748-manifests\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.820884 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.820855    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/252b401b-dddf-424b-9ff0-6c4bf5b31748-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.820984 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.820911    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/252b401b-dddf-424b-9ff0-6c4bf5b31748-tmp\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.821092 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.821075    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/252b401b-dddf-424b-9ff0-6c4bf5b31748-metrics-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.821128 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.821076    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/252b401b-dddf-424b-9ff0-6c4bf5b31748-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.851247 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.851215    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd7bx\" (UniqueName: \"kubernetes.io/projected/252b401b-dddf-424b-9ff0-6c4bf5b31748-kube-api-access-zd7bx\") pod \"opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd\" (UID: \"252b401b-dddf-424b-9ff0-6c4bf5b31748\") " pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:05.924432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:05.924401    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:06.278103 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:06.278009    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"]
Oct 30 15:43:06.292551 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:43:06.292521    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod252b401b_dddf_424b_9ff0_6c4bf5b31748.slice/crio-94a0c667c09335f3dabeead4a18d220d6e6ccb4587085d42fc8772f05c3e359a WatchSource:0}: Error finding container 94a0c667c09335f3dabeead4a18d220d6e6ccb4587085d42fc8772f05c3e359a: Status 404 returned error can't find the container with id 94a0c667c09335f3dabeead4a18d220d6e6ccb4587085d42fc8772f05c3e359a
Oct 30 15:43:06.750323 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:06.750282    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd" event={"ID":"252b401b-dddf-424b-9ff0-6c4bf5b31748","Type":"ContainerStarted","Data":"94a0c667c09335f3dabeead4a18d220d6e6ccb4587085d42fc8772f05c3e359a"}
Oct 30 15:43:09.762151 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:09.762112    4585 generic.go:356] "Generic (PLEG): container finished" podID="252b401b-dddf-424b-9ff0-6c4bf5b31748" containerID="f2091cc26a33b427f527e1ccf0f432bd4f9124097a88d3eab2de51658b35e80e" exitCode=0
Oct 30 15:43:09.762527 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:09.762176    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd" event={"ID":"252b401b-dddf-424b-9ff0-6c4bf5b31748","Type":"ContainerDied","Data":"f2091cc26a33b427f527e1ccf0f432bd4f9124097a88d3eab2de51658b35e80e"}
Oct 30 15:43:10.717080 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:10.717049    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:43:10.718597 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:10.718573    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:43:10.774721 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:10.774686    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd" event={"ID":"252b401b-dddf-424b-9ff0-6c4bf5b31748","Type":"ContainerStarted","Data":"79e1ea73661fd23ed3719ff4c6fc2a9b947980ed952921854208855711bd264c"}
Oct 30 15:43:10.775149 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:10.774823    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:10.868687 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:10.868626    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd" podStartSLOduration=2.127776801 podStartE2EDuration="5.868608382s" totalImagesPullingTime="3.258721262s" totalInitContainerRuntime="482.110319ms" isStatefulPod=false podCreationTimestamp="2026-10-30 15:43:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:43:10.856401609 +0000 UTC m=+300.292005956" watchObservedRunningTime="2026-10-30 15:43:10.868608382 +0000 UTC m=+300.304212723"
Oct 30 15:43:17.959814 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:43:17.959774    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:43:21.780060 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:21.780028    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd"
Oct 30 15:43:22.983783 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:22.983751    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"]
Oct 30 15:43:22.992115 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:22.992093    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.026875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.026849    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 30 15:43:23.027125 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.027108    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Oct 30 15:43:23.028011 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.027988    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 30 15:43:23.064264 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.064214    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b829dac-3a83-40c3-81e3-7a211e328f1a-metrics-cert\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.064264 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.064267    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0b829dac-3a83-40c3-81e3-7a211e328f1a-manager-config\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.064493 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.064418    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b829dac-3a83-40c3-81e3-7a211e328f1a-cert\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.064493 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.064459    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7jhx\" (UniqueName: \"kubernetes.io/projected/0b829dac-3a83-40c3-81e3-7a211e328f1a-kube-api-access-l7jhx\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.066992 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.066968    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-w6qjz\""
Oct 30 15:43:23.067238 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.067221    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Oct 30 15:43:23.067580 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.067564    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Oct 30 15:43:23.068004 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.067988    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"]
Oct 30 15:43:23.165557 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.165526    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b829dac-3a83-40c3-81e3-7a211e328f1a-metrics-cert\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.165557 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.165560    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0b829dac-3a83-40c3-81e3-7a211e328f1a-manager-config\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.165819 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.165589    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b829dac-3a83-40c3-81e3-7a211e328f1a-cert\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.165819 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.165614    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7jhx\" (UniqueName: \"kubernetes.io/projected/0b829dac-3a83-40c3-81e3-7a211e328f1a-kube-api-access-l7jhx\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.166252 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.166226    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0b829dac-3a83-40c3-81e3-7a211e328f1a-manager-config\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.168082 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.168058    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0b829dac-3a83-40c3-81e3-7a211e328f1a-metrics-cert\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.168180 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.168099    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b829dac-3a83-40c3-81e3-7a211e328f1a-cert\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.190120 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.190090    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7jhx\" (UniqueName: \"kubernetes.io/projected/0b829dac-3a83-40c3-81e3-7a211e328f1a-kube-api-access-l7jhx\") pod \"lws-controller-manager-7c5c6bfccb-ksrdw\" (UID: \"0b829dac-3a83-40c3-81e3-7a211e328f1a\") " pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.307057 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.306958    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:23.622601 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.622574    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"]
Oct 30 15:43:23.623879 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:43:23.623854    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b829dac_3a83_40c3_81e3_7a211e328f1a.slice/crio-90154c60823f1bd28462768deb76aeaa60de93101820770803c792479e859e31 WatchSource:0}: Error finding container 90154c60823f1bd28462768deb76aeaa60de93101820770803c792479e859e31: Status 404 returned error can't find the container with id 90154c60823f1bd28462768deb76aeaa60de93101820770803c792479e859e31
Oct 30 15:43:23.625690 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.625673    4585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 15:43:23.814854 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:23.814819    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw" event={"ID":"0b829dac-3a83-40c3-81e3-7a211e328f1a","Type":"ContainerStarted","Data":"90154c60823f1bd28462768deb76aeaa60de93101820770803c792479e859e31"}
Oct 30 15:43:26.384319 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.384243    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"]
Oct 30 15:43:26.387422 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.387403    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.419696 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.419661    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"]
Oct 30 15:43:26.495900 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.495850    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a8df42-6e87-4caf-a776-0c69b707b18a-tmp\") pod \"kube-auth-proxy-6f6967bb6d-622sf\" (UID: \"51a8df42-6e87-4caf-a776-0c69b707b18a\") " pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.495900 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.495899    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltfgj\" (UniqueName: \"kubernetes.io/projected/51a8df42-6e87-4caf-a776-0c69b707b18a-kube-api-access-ltfgj\") pod \"kube-auth-proxy-6f6967bb6d-622sf\" (UID: \"51a8df42-6e87-4caf-a776-0c69b707b18a\") " pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.496115 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.495922    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a8df42-6e87-4caf-a776-0c69b707b18a-tls-certs\") pod \"kube-auth-proxy-6f6967bb6d-622sf\" (UID: \"51a8df42-6e87-4caf-a776-0c69b707b18a\") " pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.502856 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.502788    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 15:43:26.502856 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.502830    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 15:43:26.503044 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.502891    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 15:43:26.503044 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.502985    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cxkx5\""
Oct 30 15:43:26.504078 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.504063    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 15:43:26.597380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.597329    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a8df42-6e87-4caf-a776-0c69b707b18a-tmp\") pod \"kube-auth-proxy-6f6967bb6d-622sf\" (UID: \"51a8df42-6e87-4caf-a776-0c69b707b18a\") " pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.597380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.597377    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltfgj\" (UniqueName: \"kubernetes.io/projected/51a8df42-6e87-4caf-a776-0c69b707b18a-kube-api-access-ltfgj\") pod \"kube-auth-proxy-6f6967bb6d-622sf\" (UID: \"51a8df42-6e87-4caf-a776-0c69b707b18a\") " pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.597644 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.597411    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a8df42-6e87-4caf-a776-0c69b707b18a-tls-certs\") pod \"kube-auth-proxy-6f6967bb6d-622sf\" (UID: \"51a8df42-6e87-4caf-a776-0c69b707b18a\") " pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.599765 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.599735    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a8df42-6e87-4caf-a776-0c69b707b18a-tmp\") pod \"kube-auth-proxy-6f6967bb6d-622sf\" (UID: \"51a8df42-6e87-4caf-a776-0c69b707b18a\") " pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.599896 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.599876    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a8df42-6e87-4caf-a776-0c69b707b18a-tls-certs\") pod \"kube-auth-proxy-6f6967bb6d-622sf\" (UID: \"51a8df42-6e87-4caf-a776-0c69b707b18a\") " pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.621289 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.621255    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltfgj\" (UniqueName: \"kubernetes.io/projected/51a8df42-6e87-4caf-a776-0c69b707b18a-kube-api-access-ltfgj\") pod \"kube-auth-proxy-6f6967bb6d-622sf\" (UID: \"51a8df42-6e87-4caf-a776-0c69b707b18a\") " pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.698942 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.698913    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"
Oct 30 15:43:26.831170 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.831139    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw" event={"ID":"0b829dac-3a83-40c3-81e3-7a211e328f1a","Type":"ContainerStarted","Data":"80f8a60508226d5ba5188ae3f66c2b593c89a0b95c8be3c93a43c7dfec64e6fc"}
Oct 30 15:43:26.831357 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.831278    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:26.873146 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:26.873070    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw" podStartSLOduration=2.372185997 podStartE2EDuration="4.873051818s" totalImagesPullingTime="2.500865821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:43:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:43:26.872656035 +0000 UTC m=+316.308260389" watchObservedRunningTime="2026-10-30 15:43:26.873051818 +0000 UTC m=+316.308656168"
Oct 30 15:43:27.006377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:27.006331    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf"]
Oct 30 15:43:27.007943 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:43:27.007913    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a8df42_6e87_4caf_a776_0c69b707b18a.slice/crio-28dc36673840b841e907d007fd824ef1c21ec2aa6d14821346aaf79e2e7a00c5 WatchSource:0}: Error finding container 28dc36673840b841e907d007fd824ef1c21ec2aa6d14821346aaf79e2e7a00c5: Status 404 returned error can't find the container with id 28dc36673840b841e907d007fd824ef1c21ec2aa6d14821346aaf79e2e7a00c5
Oct 30 15:43:27.837303 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:27.837254    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf" event={"ID":"51a8df42-6e87-4caf-a776-0c69b707b18a","Type":"ContainerStarted","Data":"28dc36673840b841e907d007fd824ef1c21ec2aa6d14821346aaf79e2e7a00c5"}
Oct 30 15:43:29.846424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:29.846378    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf" event={"ID":"51a8df42-6e87-4caf-a776-0c69b707b18a","Type":"ContainerStarted","Data":"0cd14656469d50f7c2a4592f214e0363f0028e9947ce169aa188759584506306"}
Oct 30 15:43:29.888971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:29.888919    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6f6967bb6d-622sf" podStartSLOduration=1.528119598 podStartE2EDuration="3.888900218s" totalImagesPullingTime="2.36078062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:43:29.887524528 +0000 UTC m=+319.323128876" watchObservedRunningTime="2026-10-30 15:43:29.888900218 +0000 UTC m=+319.324504569"
Oct 30 15:43:37.839940 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:43:37.839904    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7c5c6bfccb-ksrdw"
Oct 30 15:43:47.960470 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:43:47.960432    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:44:17.959583 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:44:17.959547    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:44:33.019511 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.019470    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-2w4j6"]
Oct 30 15:44:33.022946 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.022918    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6"
Oct 30 15:44:33.055443 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.055401    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29sw7\" (UniqueName: \"kubernetes.io/projected/0b99e03c-dfa7-49c7-8899-e8751735da37-kube-api-access-29sw7\") pod \"model-serving-api-75bd89fdfc-2w4j6\" (UID: \"0b99e03c-dfa7-49c7-8899-e8751735da37\") " pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6"
Oct 30 15:44:33.055824 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.055801    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b99e03c-dfa7-49c7-8899-e8751735da37-tls-certs\") pod \"model-serving-api-75bd89fdfc-2w4j6\" (UID: \"0b99e03c-dfa7-49c7-8899-e8751735da37\") " pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6"
Oct 30 15:44:33.057387 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.057360    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-2w4j6"]
Oct 30 15:44:33.104834 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.104804    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-6m6gf\""
Oct 30 15:44:33.104834 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.104814    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 30 15:44:33.157112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.157072    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29sw7\" (UniqueName: \"kubernetes.io/projected/0b99e03c-dfa7-49c7-8899-e8751735da37-kube-api-access-29sw7\") pod \"model-serving-api-75bd89fdfc-2w4j6\" (UID: \"0b99e03c-dfa7-49c7-8899-e8751735da37\") " pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6"
Oct 30 15:44:33.157286 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.157143    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b99e03c-dfa7-49c7-8899-e8751735da37-tls-certs\") pod \"model-serving-api-75bd89fdfc-2w4j6\" (UID: \"0b99e03c-dfa7-49c7-8899-e8751735da37\") " pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6"
Oct 30 15:44:33.159793 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.159767    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b99e03c-dfa7-49c7-8899-e8751735da37-tls-certs\") pod \"model-serving-api-75bd89fdfc-2w4j6\" (UID: \"0b99e03c-dfa7-49c7-8899-e8751735da37\") " pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6"
Oct 30 15:44:33.182421 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.182388    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29sw7\" (UniqueName: \"kubernetes.io/projected/0b99e03c-dfa7-49c7-8899-e8751735da37-kube-api-access-29sw7\") pod \"model-serving-api-75bd89fdfc-2w4j6\" (UID: \"0b99e03c-dfa7-49c7-8899-e8751735da37\") " pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6"
Oct 30 15:44:33.335411 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.335301    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6"
Oct 30 15:44:33.632920 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:33.632887    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-2w4j6"]
Oct 30 15:44:33.633979 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:44:33.633957    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b99e03c_dfa7_49c7_8899_e8751735da37.slice/crio-5bab21091d3623de6b38ea03467786c5d77831edaeca75709769f31d054826a7 WatchSource:0}: Error finding container 5bab21091d3623de6b38ea03467786c5d77831edaeca75709769f31d054826a7: Status 404 returned error can't find the container with id 5bab21091d3623de6b38ea03467786c5d77831edaeca75709769f31d054826a7
Oct 30 15:44:34.062617 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:34.062577    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6" event={"ID":"0b99e03c-dfa7-49c7-8899-e8751735da37","Type":"ContainerStarted","Data":"5bab21091d3623de6b38ea03467786c5d77831edaeca75709769f31d054826a7"}
Oct 30 15:44:36.073012 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:36.072975    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6" event={"ID":"0b99e03c-dfa7-49c7-8899-e8751735da37","Type":"ContainerStarted","Data":"ec2ad6a318184ff6c186fdf901d0f6aa89e70876c7bb87894d8c82df1892a46d"}
Oct 30 15:44:36.073502 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:36.073088    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6"
Oct 30 15:44:36.115775 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:36.115724    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6" podStartSLOduration=2.6806167690000002 podStartE2EDuration="4.115709398s" totalImagesPullingTime="1.435092629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:44:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:44:36.114990251 +0000 UTC m=+385.550594598" watchObservedRunningTime="2026-10-30 15:44:36.115709398 +0000 UTC m=+385.551313746"
Oct 30 15:44:47.081404 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:44:47.081365    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-2w4j6"
Oct 30 15:44:47.960288 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:44:47.960230    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:45:17.961005 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:45:17.960969    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:45:43.893111 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:43.893074    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"]
Oct 30 15:45:43.896542 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:43.896517    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:45:43.914775 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:43.914747    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2fsr\" (UniqueName: \"kubernetes.io/projected/0d90c745-3a9d-466b-8df8-266c19ae2b2e-kube-api-access-j2fsr\") pod \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" (UID: \"0d90c745-3a9d-466b-8df8-266c19ae2b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:45:43.914891 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:43.914784    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d90c745-3a9d-466b-8df8-266c19ae2b2e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" (UID: \"0d90c745-3a9d-466b-8df8-266c19ae2b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:45:43.919362 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:43.919328    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"]
Oct 30 15:45:43.930508 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:43.930472    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 15:45:43.930508 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:43.930496    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 15:45:43.930686 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:43.930575    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-trpmg\""
Oct 30 15:45:44.015191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:44.015150    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2fsr\" (UniqueName: \"kubernetes.io/projected/0d90c745-3a9d-466b-8df8-266c19ae2b2e-kube-api-access-j2fsr\") pod \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" (UID: \"0d90c745-3a9d-466b-8df8-266c19ae2b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:45:44.015191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:44.015190    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d90c745-3a9d-466b-8df8-266c19ae2b2e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" (UID: \"0d90c745-3a9d-466b-8df8-266c19ae2b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:45:44.015540 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:44.015521    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d90c745-3a9d-466b-8df8-266c19ae2b2e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" (UID: \"0d90c745-3a9d-466b-8df8-266c19ae2b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:45:44.043886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:44.043849    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2fsr\" (UniqueName: \"kubernetes.io/projected/0d90c745-3a9d-466b-8df8-266c19ae2b2e-kube-api-access-j2fsr\") pod \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" (UID: \"0d90c745-3a9d-466b-8df8-266c19ae2b2e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:45:44.209129 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:44.209080    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:45:44.541212 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:44.541166    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"]
Oct 30 15:45:44.544129 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:45:44.544100    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d90c745_3a9d_466b_8df8_266c19ae2b2e.slice/crio-f7da80e16632248e7d527c47bd8cc02f1f979b82b3640fe8fa1d245fc5b7ab57 WatchSource:0}: Error finding container f7da80e16632248e7d527c47bd8cc02f1f979b82b3640fe8fa1d245fc5b7ab57: Status 404 returned error can't find the container with id f7da80e16632248e7d527c47bd8cc02f1f979b82b3640fe8fa1d245fc5b7ab57
Oct 30 15:45:45.312231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:45.312199    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn" event={"ID":"0d90c745-3a9d-466b-8df8-266c19ae2b2e","Type":"ContainerStarted","Data":"f7da80e16632248e7d527c47bd8cc02f1f979b82b3640fe8fa1d245fc5b7ab57"}
Oct 30 15:45:47.965290 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:45:47.965246    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:45:48.329112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:48.329002    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn" event={"ID":"0d90c745-3a9d-466b-8df8-266c19ae2b2e","Type":"ContainerStarted","Data":"b5f7659c0bceced03bfeb93503825692dcf875472f16a0a5cc191ffac29bece0"}
Oct 30 15:45:48.329283 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:48.329118    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:45:48.401588 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:48.401536    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn" podStartSLOduration=1.7981270070000002 podStartE2EDuration="5.401521983s" totalImagesPullingTime="3.603394976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:45:48.388996713 +0000 UTC m=+457.824601075" watchObservedRunningTime="2026-10-30 15:45:48.401521983 +0000 UTC m=+457.837126331"
Oct 30 15:45:59.335769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:45:59.335732    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:46:03.174652 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.174618    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"]
Oct 30 15:46:03.175121 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.174891    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn" podUID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" containerName="manager" containerID="cri-o://b5f7659c0bceced03bfeb93503825692dcf875472f16a0a5cc191ffac29bece0" gracePeriod=2
Oct 30 15:46:03.175936 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.175876    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"]
Oct 30 15:46:03.188870 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:46:03.188833    4585 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" is forbidden: User \"system:node:ip-10-0-140-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-110.ec2.internal' and this object" podUID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:46:03.190462 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.190439    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"]
Oct 30 15:46:03.190741 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.190729    4585 cpu_manager.go:414] "containerMap: removing container" podUID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" containerName="manager"
Oct 30 15:46:03.190782 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.190743    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" containerName="manager"
Oct 30 15:46:03.190815 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.190791    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" containerName="manager"
Oct 30 15:46:03.193759 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.193737    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:03.208987 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:46:03.208958    4585 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" is forbidden: User \"system:node:ip-10-0-140-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-110.ec2.internal' and this object" podUID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:46:03.229670 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.229642    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"]
Oct 30 15:46:03.272397 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.272365    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d45b17f-2aba-467a-87f6-53e14db94f2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-4wb27\" (UID: \"1d45b17f-2aba-467a-87f6-53e14db94f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:03.272599 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.272510    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx2km\" (UniqueName: \"kubernetes.io/projected/1d45b17f-2aba-467a-87f6-53e14db94f2c-kube-api-access-jx2km\") pod \"kuadrant-operator-controller-manager-6b6fb56846-4wb27\" (UID: \"1d45b17f-2aba-467a-87f6-53e14db94f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:03.373946 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.373905    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d45b17f-2aba-467a-87f6-53e14db94f2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-4wb27\" (UID: \"1d45b17f-2aba-467a-87f6-53e14db94f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:03.374093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.374046    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx2km\" (UniqueName: \"kubernetes.io/projected/1d45b17f-2aba-467a-87f6-53e14db94f2c-kube-api-access-jx2km\") pod \"kuadrant-operator-controller-manager-6b6fb56846-4wb27\" (UID: \"1d45b17f-2aba-467a-87f6-53e14db94f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:03.374378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.374331    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d45b17f-2aba-467a-87f6-53e14db94f2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-4wb27\" (UID: \"1d45b17f-2aba-467a-87f6-53e14db94f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:03.387302 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.387268    4585 generic.go:356] "Generic (PLEG): container finished" podID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" containerID="b5f7659c0bceced03bfeb93503825692dcf875472f16a0a5cc191ffac29bece0" exitCode=0
Oct 30 15:46:03.399852 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.399820    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx2km\" (UniqueName: \"kubernetes.io/projected/1d45b17f-2aba-467a-87f6-53e14db94f2c-kube-api-access-jx2km\") pod \"kuadrant-operator-controller-manager-6b6fb56846-4wb27\" (UID: \"1d45b17f-2aba-467a-87f6-53e14db94f2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:03.432921 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.432868    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:46:03.446380 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:46:03.446332    4585 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" is forbidden: User \"system:node:ip-10-0-140-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-110.ec2.internal' and this object" podUID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:46:03.474628 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.474605    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d90c745-3a9d-466b-8df8-266c19ae2b2e-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d90c745-3a9d-466b-8df8-266c19ae2b2e-extensions-socket-volume\") pod \"0d90c745-3a9d-466b-8df8-266c19ae2b2e\" (UID: \"0d90c745-3a9d-466b-8df8-266c19ae2b2e\") "
Oct 30 15:46:03.474736 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.474658    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d90c745-3a9d-466b-8df8-266c19ae2b2e-kube-api-access-j2fsr\" (UniqueName: \"kubernetes.io/projected/0d90c745-3a9d-466b-8df8-266c19ae2b2e-kube-api-access-j2fsr\") pod \"0d90c745-3a9d-466b-8df8-266c19ae2b2e\" (UID: \"0d90c745-3a9d-466b-8df8-266c19ae2b2e\") "
Oct 30 15:46:03.475231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.475201    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d90c745-3a9d-466b-8df8-266c19ae2b2e-extensions-socket-volume" pod "0d90c745-3a9d-466b-8df8-266c19ae2b2e" (UID: "0d90c745-3a9d-466b-8df8-266c19ae2b2e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:46:03.476863 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.476838    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d90c745-3a9d-466b-8df8-266c19ae2b2e-kube-api-access-j2fsr" pod "0d90c745-3a9d-466b-8df8-266c19ae2b2e" (UID: "0d90c745-3a9d-466b-8df8-266c19ae2b2e"). InnerVolumeSpecName "kube-api-access-j2fsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:46:03.548800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.548765    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:03.576033 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.576004    4585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d90c745-3a9d-466b-8df8-266c19ae2b2e-extensions-socket-volume\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:46:03.576033 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.576035    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2fsr\" (UniqueName: \"kubernetes.io/projected/0d90c745-3a9d-466b-8df8-266c19ae2b2e-kube-api-access-j2fsr\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:46:03.878365 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:03.878271    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"]
Oct 30 15:46:03.891809 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:46:03.891782    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d45b17f_2aba_467a_87f6_53e14db94f2c.slice/crio-ba28b99dc6caadc4f42054cbe2345b4cbe0eb6383c83aa51a6d9def172dc22be WatchSource:0}: Error finding container ba28b99dc6caadc4f42054cbe2345b4cbe0eb6383c83aa51a6d9def172dc22be: Status 404 returned error can't find the container with id ba28b99dc6caadc4f42054cbe2345b4cbe0eb6383c83aa51a6d9def172dc22be
Oct 30 15:46:04.392793 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:04.392762    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:46:04.393257 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:04.392770    4585 scope.go:122] "RemoveContainer" containerID="b5f7659c0bceced03bfeb93503825692dcf875472f16a0a5cc191ffac29bece0"
Oct 30 15:46:04.394602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:04.394579    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27" event={"ID":"1d45b17f-2aba-467a-87f6-53e14db94f2c","Type":"ContainerStarted","Data":"94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742"}
Oct 30 15:46:04.394675 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:04.394627    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27" event={"ID":"1d45b17f-2aba-467a-87f6-53e14db94f2c","Type":"ContainerStarted","Data":"ba28b99dc6caadc4f42054cbe2345b4cbe0eb6383c83aa51a6d9def172dc22be"}
Oct 30 15:46:04.394749 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:04.394727    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:04.406770 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:46:04.406746    4585 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" is forbidden: User \"system:node:ip-10-0-140-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-110.ec2.internal' and this object" podUID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:46:04.420294 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:46:04.420273    4585 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-7jhsn\" is forbidden: User \"system:node:ip-10-0-140-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-110.ec2.internal' and this object" podUID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-7jhsn"
Oct 30 15:46:04.518542 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:04.518489    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27" podStartSLOduration=1.518474323 podStartE2EDuration="1.518474323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:46:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:46:04.507006059 +0000 UTC m=+473.942610407" watchObservedRunningTime="2026-10-30 15:46:04.518474323 +0000 UTC m=+473.954078667"
Oct 30 15:46:04.699277 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:04.699240    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d90c745-3a9d-466b-8df8-266c19ae2b2e" path="/var/lib/kubelet/pods/0d90c745-3a9d-466b-8df8-266c19ae2b2e/volumes"
Oct 30 15:46:15.401463 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:15.401429    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:17.959738 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:46:17.959705    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:46:18.711304 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:18.711269    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"]
Oct 30 15:46:18.711551 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:18.711516    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27" podUID="1d45b17f-2aba-467a-87f6-53e14db94f2c" containerName="manager" containerID="cri-o://94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742" gracePeriod=10
Oct 30 15:46:18.922215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:18.922186    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"]
Oct 30 15:46:18.924411 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:18.924394    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 15:46:18.950713 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:18.950685    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"]
Oct 30 15:46:18.957014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:18.956993    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:19.008838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.008759    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nphf8\" (UniqueName: \"kubernetes.io/projected/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-kube-api-access-nphf8\") pod \"kuadrant-operator-controller-manager-57bb9bfff-ptltr\" (UID: \"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 15:46:19.008838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.008808    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-ptltr\" (UID: \"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 15:46:19.109138 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.109104    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d45b17f-2aba-467a-87f6-53e14db94f2c-kube-api-access-jx2km\" (UniqueName: \"kubernetes.io/projected/1d45b17f-2aba-467a-87f6-53e14db94f2c-kube-api-access-jx2km\") pod \"1d45b17f-2aba-467a-87f6-53e14db94f2c\" (UID: \"1d45b17f-2aba-467a-87f6-53e14db94f2c\") "
Oct 30 15:46:19.109299 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.109153    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d45b17f-2aba-467a-87f6-53e14db94f2c-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d45b17f-2aba-467a-87f6-53e14db94f2c-extensions-socket-volume\") pod \"1d45b17f-2aba-467a-87f6-53e14db94f2c\" (UID: \"1d45b17f-2aba-467a-87f6-53e14db94f2c\") "
Oct 30 15:46:19.109341 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.109313    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nphf8\" (UniqueName: \"kubernetes.io/projected/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-kube-api-access-nphf8\") pod \"kuadrant-operator-controller-manager-57bb9bfff-ptltr\" (UID: \"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 15:46:19.109395 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.109369    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-ptltr\" (UID: \"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 15:46:19.109609 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.109580    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d45b17f-2aba-467a-87f6-53e14db94f2c-extensions-socket-volume" pod "1d45b17f-2aba-467a-87f6-53e14db94f2c" (UID: "1d45b17f-2aba-467a-87f6-53e14db94f2c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:46:19.109741 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.109722    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-ptltr\" (UID: \"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 15:46:19.111177 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.111150    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d45b17f-2aba-467a-87f6-53e14db94f2c-kube-api-access-jx2km" pod "1d45b17f-2aba-467a-87f6-53e14db94f2c" (UID: "1d45b17f-2aba-467a-87f6-53e14db94f2c"). InnerVolumeSpecName "kube-api-access-jx2km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:46:19.133395 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.133372    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nphf8\" (UniqueName: \"kubernetes.io/projected/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-kube-api-access-nphf8\") pod \"kuadrant-operator-controller-manager-57bb9bfff-ptltr\" (UID: \"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 15:46:19.210407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.210374    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx2km\" (UniqueName: \"kubernetes.io/projected/1d45b17f-2aba-467a-87f6-53e14db94f2c-kube-api-access-jx2km\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:46:19.210407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.210403    4585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d45b17f-2aba-467a-87f6-53e14db94f2c-extensions-socket-volume\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:46:19.236316 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.236285    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 15:46:19.454707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.454674    4585 generic.go:356] "Generic (PLEG): container finished" podID="1d45b17f-2aba-467a-87f6-53e14db94f2c" containerID="94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742" exitCode=0
Oct 30 15:46:19.454875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.454717    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"
Oct 30 15:46:19.454875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.454756    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27" event={"ID":"1d45b17f-2aba-467a-87f6-53e14db94f2c","Type":"ContainerDied","Data":"94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742"}
Oct 30 15:46:19.454875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.454801    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27" event={"ID":"1d45b17f-2aba-467a-87f6-53e14db94f2c","Type":"ContainerDied","Data":"ba28b99dc6caadc4f42054cbe2345b4cbe0eb6383c83aa51a6d9def172dc22be"}
Oct 30 15:46:19.454875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.454826    4585 scope.go:122] "RemoveContainer" containerID="94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742"
Oct 30 15:46:19.463744 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.463723    4585 scope.go:122] "RemoveContainer" containerID="94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742"
Oct 30 15:46:19.464021 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:46:19.464003    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742\": container with ID starting with 94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742 not found: ID does not exist" containerID="94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742"
Oct 30 15:46:19.464067 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.464032    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742"} err="failed to get container status \"94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742\": rpc error: code = NotFound desc = could not find container \"94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742\": container with ID starting with 94ba702d88c78ca2c220ad1e1a8dc5b5b7f61981679c3452005b698cd9cd4742 not found: ID does not exist"
Oct 30 15:46:19.523295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.523259    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"]
Oct 30 15:46:19.525690 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.525666    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-4wb27"]
Oct 30 15:46:19.573071 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:19.573043    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"]
Oct 30 15:46:19.574563 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:46:19.574538    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a4ab6c7_0731_4ea1_84dd_b8aa22e998c1.slice/crio-efe2a86d974ca6abd9e7476c53c33377b5b3ab727ef59f0fdec7258a990b0bde WatchSource:0}: Error finding container efe2a86d974ca6abd9e7476c53c33377b5b3ab727ef59f0fdec7258a990b0bde: Status 404 returned error can't find the container with id efe2a86d974ca6abd9e7476c53c33377b5b3ab727ef59f0fdec7258a990b0bde
Oct 30 15:46:20.459717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:20.459685    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr" event={"ID":"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1","Type":"ContainerStarted","Data":"38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1"}
Oct 30 15:46:20.460139 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:20.459749    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 15:46:20.460139 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:20.459769    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr" event={"ID":"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1","Type":"ContainerStarted","Data":"efe2a86d974ca6abd9e7476c53c33377b5b3ab727ef59f0fdec7258a990b0bde"}
Oct 30 15:46:20.532887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:20.532833    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr" podStartSLOduration=2.53281843 podStartE2EDuration="2.53281843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:46:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:46:20.532648391 +0000 UTC m=+489.968252753" watchObservedRunningTime="2026-10-30 15:46:20.53281843 +0000 UTC m=+489.968422778"
Oct 30 15:46:20.699500 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:20.699471    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d45b17f-2aba-467a-87f6-53e14db94f2c" path="/var/lib/kubelet/pods/1d45b17f-2aba-467a-87f6-53e14db94f2c/volumes"
Oct 30 15:46:31.466402 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:31.466317    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 15:46:47.960797 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:46:47.960758    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:46:51.504798 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.504762    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-5zbpz"]
Oct 30 15:46:51.505214 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.505053    4585 cpu_manager.go:414] "containerMap: removing container" podUID="1d45b17f-2aba-467a-87f6-53e14db94f2c" containerName="manager"
Oct 30 15:46:51.505214 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.505065    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d45b17f-2aba-467a-87f6-53e14db94f2c" containerName="manager"
Oct 30 15:46:51.505214 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.505130    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d45b17f-2aba-467a-87f6-53e14db94f2c" containerName="manager"
Oct 30 15:46:51.506996 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.506969    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-5zbpz"
Oct 30 15:46:51.526961 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.526927    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-5zbpz"]
Oct 30 15:46:51.534576 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.534552    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jg68v\""
Oct 30 15:46:51.578944 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.578910    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt2bq\" (UniqueName: \"kubernetes.io/projected/fa431760-bb5b-4a2c-a55e-f7f236e91b14-kube-api-access-tt2bq\") pod \"authorino-57f7f8659f-5zbpz\" (UID: \"fa431760-bb5b-4a2c-a55e-f7f236e91b14\") " pod="kuadrant-system/authorino-57f7f8659f-5zbpz"
Oct 30 15:46:51.680198 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.680156    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt2bq\" (UniqueName: \"kubernetes.io/projected/fa431760-bb5b-4a2c-a55e-f7f236e91b14-kube-api-access-tt2bq\") pod \"authorino-57f7f8659f-5zbpz\" (UID: \"fa431760-bb5b-4a2c-a55e-f7f236e91b14\") " pod="kuadrant-system/authorino-57f7f8659f-5zbpz"
Oct 30 15:46:51.704007 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.703976    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt2bq\" (UniqueName: \"kubernetes.io/projected/fa431760-bb5b-4a2c-a55e-f7f236e91b14-kube-api-access-tt2bq\") pod \"authorino-57f7f8659f-5zbpz\" (UID: \"fa431760-bb5b-4a2c-a55e-f7f236e91b14\") " pod="kuadrant-system/authorino-57f7f8659f-5zbpz"
Oct 30 15:46:51.835045 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:51.834962    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-5zbpz"
Oct 30 15:46:52.131431 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:52.131405    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-5zbpz"]
Oct 30 15:46:52.132911 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:46:52.132883    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa431760_bb5b_4a2c_a55e_f7f236e91b14.slice/crio-7e99ff942e3c6c983c9d8fe6daf053fb07af903fa82d5de58ab16be054acf2f0 WatchSource:0}: Error finding container 7e99ff942e3c6c983c9d8fe6daf053fb07af903fa82d5de58ab16be054acf2f0: Status 404 returned error can't find the container with id 7e99ff942e3c6c983c9d8fe6daf053fb07af903fa82d5de58ab16be054acf2f0
Oct 30 15:46:52.144382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:52.144329    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wsvw2"]
Oct 30 15:46:52.150297 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:52.150266    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2"
Oct 30 15:46:52.166029 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:52.166000    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wsvw2"]
Oct 30 15:46:52.184447 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:52.184414    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfm6z\" (UniqueName: \"kubernetes.io/projected/16e5eca1-86b2-4629-8ea0-f0bec633f314-kube-api-access-jfm6z\") pod \"authorino-56c6c9ddd5-wsvw2\" (UID: \"16e5eca1-86b2-4629-8ea0-f0bec633f314\") " pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2"
Oct 30 15:46:52.285234 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:52.285192    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfm6z\" (UniqueName: \"kubernetes.io/projected/16e5eca1-86b2-4629-8ea0-f0bec633f314-kube-api-access-jfm6z\") pod \"authorino-56c6c9ddd5-wsvw2\" (UID: \"16e5eca1-86b2-4629-8ea0-f0bec633f314\") " pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2"
Oct 30 15:46:52.308474 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:52.308441    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfm6z\" (UniqueName: \"kubernetes.io/projected/16e5eca1-86b2-4629-8ea0-f0bec633f314-kube-api-access-jfm6z\") pod \"authorino-56c6c9ddd5-wsvw2\" (UID: \"16e5eca1-86b2-4629-8ea0-f0bec633f314\") " pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2"
Oct 30 15:46:52.461805 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:52.461770    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2"
Oct 30 15:46:52.585727 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:52.585688    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-5zbpz" event={"ID":"fa431760-bb5b-4a2c-a55e-f7f236e91b14","Type":"ContainerStarted","Data":"7e99ff942e3c6c983c9d8fe6daf053fb07af903fa82d5de58ab16be054acf2f0"}
Oct 30 15:46:52.754635 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:52.754607    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wsvw2"]
Oct 30 15:46:52.756284 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:46:52.756257    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16e5eca1_86b2_4629_8ea0_f0bec633f314.slice/crio-cd822e447fa679371c3e56b61410db81962f6e4344e6f03c220f2600afcf6058 WatchSource:0}: Error finding container cd822e447fa679371c3e56b61410db81962f6e4344e6f03c220f2600afcf6058: Status 404 returned error can't find the container with id cd822e447fa679371c3e56b61410db81962f6e4344e6f03c220f2600afcf6058
Oct 30 15:46:53.591583 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:53.591527    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2" event={"ID":"16e5eca1-86b2-4629-8ea0-f0bec633f314","Type":"ContainerStarted","Data":"cd822e447fa679371c3e56b61410db81962f6e4344e6f03c220f2600afcf6058"}
Oct 30 15:46:54.597408 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:54.597370    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2" event={"ID":"16e5eca1-86b2-4629-8ea0-f0bec633f314","Type":"ContainerStarted","Data":"2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4"}
Oct 30 15:46:54.598886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:54.598856    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-5zbpz" event={"ID":"fa431760-bb5b-4a2c-a55e-f7f236e91b14","Type":"ContainerStarted","Data":"1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942"}
Oct 30 15:46:54.636409 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:54.636321    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2" podStartSLOduration=1.259164171 podStartE2EDuration="2.636302561s" totalImagesPullingTime="1.37713839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:46:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:46:54.635826356 +0000 UTC m=+524.071430704" watchObservedRunningTime="2026-10-30 15:46:54.636302561 +0000 UTC m=+524.071906911"
Oct 30 15:46:54.661769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:54.661738    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-5zbpz"]
Oct 30 15:46:54.673792 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:54.673747    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-5zbpz" podStartSLOduration=1.839042685 podStartE2EDuration="3.673732709s" totalImagesPullingTime="1.834690024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:46:54.673075418 +0000 UTC m=+524.108679767" watchObservedRunningTime="2026-10-30 15:46:54.673732709 +0000 UTC m=+524.109337057"
Oct 30 15:46:56.609657 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:56.609619    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-5zbpz" podUID="fa431760-bb5b-4a2c-a55e-f7f236e91b14" containerName="authorino" containerID="cri-o://1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942" gracePeriod=30
Oct 30 15:46:56.856731 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:56.856702    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-5zbpz"
Oct 30 15:46:56.927146 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:56.927116    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa431760-bb5b-4a2c-a55e-f7f236e91b14-kube-api-access-tt2bq\" (UniqueName: \"kubernetes.io/projected/fa431760-bb5b-4a2c-a55e-f7f236e91b14-kube-api-access-tt2bq\") pod \"fa431760-bb5b-4a2c-a55e-f7f236e91b14\" (UID: \"fa431760-bb5b-4a2c-a55e-f7f236e91b14\") "
Oct 30 15:46:56.929155 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:56.929125    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa431760-bb5b-4a2c-a55e-f7f236e91b14-kube-api-access-tt2bq" pod "fa431760-bb5b-4a2c-a55e-f7f236e91b14" (UID: "fa431760-bb5b-4a2c-a55e-f7f236e91b14"). InnerVolumeSpecName "kube-api-access-tt2bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:46:57.028544 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:57.028507    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt2bq\" (UniqueName: \"kubernetes.io/projected/fa431760-bb5b-4a2c-a55e-f7f236e91b14-kube-api-access-tt2bq\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:46:57.616504 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:57.616468    4585 generic.go:356] "Generic (PLEG): container finished" podID="fa431760-bb5b-4a2c-a55e-f7f236e91b14" containerID="1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942" exitCode=0
Oct 30 15:46:57.616504 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:57.616510    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-5zbpz"
Oct 30 15:46:57.616988 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:57.616510    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-5zbpz" event={"ID":"fa431760-bb5b-4a2c-a55e-f7f236e91b14","Type":"ContainerDied","Data":"1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942"}
Oct 30 15:46:57.616988 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:57.616625    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-5zbpz" event={"ID":"fa431760-bb5b-4a2c-a55e-f7f236e91b14","Type":"ContainerDied","Data":"7e99ff942e3c6c983c9d8fe6daf053fb07af903fa82d5de58ab16be054acf2f0"}
Oct 30 15:46:57.616988 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:57.616645    4585 scope.go:122] "RemoveContainer" containerID="1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942"
Oct 30 15:46:57.624838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:57.624818    4585 scope.go:122] "RemoveContainer" containerID="1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942"
Oct 30 15:46:57.625138 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:46:57.625118    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942\": container with ID starting with 1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942 not found: ID does not exist" containerID="1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942"
Oct 30 15:46:57.625192 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:57.625149    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942"} err="failed to get container status \"1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942\": rpc error: code = NotFound desc = could not find container \"1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942\": container with ID starting with 1c8361eda8d10c133932c7bb9d458b49833f7f64b278df73ddd8503aff7ab942 not found: ID does not exist"
Oct 30 15:46:57.671759 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:57.671728    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-5zbpz"]
Oct 30 15:46:57.675226 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:57.675198    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-5zbpz"]
Oct 30 15:46:58.699894 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:46:58.699863    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa431760-bb5b-4a2c-a55e-f7f236e91b14" path="/var/lib/kubelet/pods/fa431760-bb5b-4a2c-a55e-f7f236e91b14/volumes"
Oct 30 15:47:17.960918 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:47:17.960882    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:47:24.210169 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.210131    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-r6xxv"]
Oct 30 15:47:24.210818 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.210613    4585 cpu_manager.go:414] "containerMap: removing container" podUID="fa431760-bb5b-4a2c-a55e-f7f236e91b14" containerName="authorino"
Oct 30 15:47:24.210818 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.210634    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa431760-bb5b-4a2c-a55e-f7f236e91b14" containerName="authorino"
Oct 30 15:47:24.210818 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.210739    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa431760-bb5b-4a2c-a55e-f7f236e91b14" containerName="authorino"
Oct 30 15:47:24.216176 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.216151    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-r6xxv"
Oct 30 15:47:24.230627 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.230601    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-r6xxv"]
Oct 30 15:47:24.366486 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.366452    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z48h4\" (UniqueName: \"kubernetes.io/projected/5cad47cc-741c-4577-928f-00a391988f06-kube-api-access-z48h4\") pod \"authorino-6f6944bcdd-r6xxv\" (UID: \"5cad47cc-741c-4577-928f-00a391988f06\") " pod="kuadrant-system/authorino-6f6944bcdd-r6xxv"
Oct 30 15:47:24.467717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.467634    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z48h4\" (UniqueName: \"kubernetes.io/projected/5cad47cc-741c-4577-928f-00a391988f06-kube-api-access-z48h4\") pod \"authorino-6f6944bcdd-r6xxv\" (UID: \"5cad47cc-741c-4577-928f-00a391988f06\") " pod="kuadrant-system/authorino-6f6944bcdd-r6xxv"
Oct 30 15:47:24.473425 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.473398    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-r6xxv"]
Oct 30 15:47:24.473601 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:47:24.473583    4585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-z48h4], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f6944bcdd-r6xxv" podUID="5cad47cc-741c-4577-928f-00a391988f06"
Oct 30 15:47:24.523838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.523808    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z48h4\" (UniqueName: \"kubernetes.io/projected/5cad47cc-741c-4577-928f-00a391988f06-kube-api-access-z48h4\") pod \"authorino-6f6944bcdd-r6xxv\" (UID: \"5cad47cc-741c-4577-928f-00a391988f06\") " pod="kuadrant-system/authorino-6f6944bcdd-r6xxv"
Oct 30 15:47:24.525067 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.525044    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-szsfk"]
Oct 30 15:47:24.528809 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.528787    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-szsfk"
Oct 30 15:47:24.546256 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.546226    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 15:47:24.547302 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.547276    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-szsfk"]
Oct 30 15:47:24.618990 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.618962    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-szsfk"]
Oct 30 15:47:24.619177 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:47:24.619159    4585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rm2d6 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6889cbd8c4-szsfk" podUID="ffebfd77-3636-477f-97e7-50f8ee46f040"
Oct 30 15:47:24.649143 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.649116    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6ddd779586-6tngq"]
Oct 30 15:47:24.652714 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.652698    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddd779586-6tngq"
Oct 30 15:47:24.669021 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.668995    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ffebfd77-3636-477f-97e7-50f8ee46f040-tls-cert\") pod \"authorino-6889cbd8c4-szsfk\" (UID: \"ffebfd77-3636-477f-97e7-50f8ee46f040\") " pod="kuadrant-system/authorino-6889cbd8c4-szsfk"
Oct 30 15:47:24.669138 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.669040    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm2d6\" (UniqueName: \"kubernetes.io/projected/ffebfd77-3636-477f-97e7-50f8ee46f040-kube-api-access-rm2d6\") pod \"authorino-6889cbd8c4-szsfk\" (UID: \"ffebfd77-3636-477f-97e7-50f8ee46f040\") " pod="kuadrant-system/authorino-6889cbd8c4-szsfk"
Oct 30 15:47:24.672914 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.672890    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6ddd779586-6tngq"]
Oct 30 15:47:24.716464 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.716433    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-szsfk"
Oct 30 15:47:24.716636 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.716433    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-r6xxv"
Oct 30 15:47:24.723426 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.723366    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-szsfk"
Oct 30 15:47:24.726085 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.726063    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-r6xxv"
Oct 30 15:47:24.770397 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.770343    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn8rj\" (UniqueName: \"kubernetes.io/projected/3783d21c-aa91-4c4c-abe5-402f6e03da90-kube-api-access-zn8rj\") pod \"authorino-6ddd779586-6tngq\" (UID: \"3783d21c-aa91-4c4c-abe5-402f6e03da90\") " pod="kuadrant-system/authorino-6ddd779586-6tngq"
Oct 30 15:47:24.770559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.770457    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ffebfd77-3636-477f-97e7-50f8ee46f040-tls-cert\") pod \"authorino-6889cbd8c4-szsfk\" (UID: \"ffebfd77-3636-477f-97e7-50f8ee46f040\") " pod="kuadrant-system/authorino-6889cbd8c4-szsfk"
Oct 30 15:47:24.770559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.770477    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3783d21c-aa91-4c4c-abe5-402f6e03da90-tls-cert\") pod \"authorino-6ddd779586-6tngq\" (UID: \"3783d21c-aa91-4c4c-abe5-402f6e03da90\") " pod="kuadrant-system/authorino-6ddd779586-6tngq"
Oct 30 15:47:24.770559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.770527    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm2d6\" (UniqueName: \"kubernetes.io/projected/ffebfd77-3636-477f-97e7-50f8ee46f040-kube-api-access-rm2d6\") pod \"authorino-6889cbd8c4-szsfk\" (UID: \"ffebfd77-3636-477f-97e7-50f8ee46f040\") " pod="kuadrant-system/authorino-6889cbd8c4-szsfk"
Oct 30 15:47:24.772861 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.772834    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ffebfd77-3636-477f-97e7-50f8ee46f040-tls-cert\") pod \"authorino-6889cbd8c4-szsfk\" (UID: \"ffebfd77-3636-477f-97e7-50f8ee46f040\") " pod="kuadrant-system/authorino-6889cbd8c4-szsfk"
Oct 30 15:47:24.793881 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.793852    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm2d6\" (UniqueName: \"kubernetes.io/projected/ffebfd77-3636-477f-97e7-50f8ee46f040-kube-api-access-rm2d6\") pod \"authorino-6889cbd8c4-szsfk\" (UID: \"ffebfd77-3636-477f-97e7-50f8ee46f040\") " pod="kuadrant-system/authorino-6889cbd8c4-szsfk"
Oct 30 15:47:24.871843 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.871811    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ffebfd77-3636-477f-97e7-50f8ee46f040-tls-cert\" (UniqueName: \"kubernetes.io/secret/ffebfd77-3636-477f-97e7-50f8ee46f040-tls-cert\") pod \"ffebfd77-3636-477f-97e7-50f8ee46f040\" (UID: \"ffebfd77-3636-477f-97e7-50f8ee46f040\") "
Oct 30 15:47:24.871843 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.871849    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5cad47cc-741c-4577-928f-00a391988f06-kube-api-access-z48h4\" (UniqueName: \"kubernetes.io/projected/5cad47cc-741c-4577-928f-00a391988f06-kube-api-access-z48h4\") pod \"5cad47cc-741c-4577-928f-00a391988f06\" (UID: \"5cad47cc-741c-4577-928f-00a391988f06\") "
Oct 30 15:47:24.872080 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.871958    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn8rj\" (UniqueName: \"kubernetes.io/projected/3783d21c-aa91-4c4c-abe5-402f6e03da90-kube-api-access-zn8rj\") pod \"authorino-6ddd779586-6tngq\" (UID: \"3783d21c-aa91-4c4c-abe5-402f6e03da90\") " pod="kuadrant-system/authorino-6ddd779586-6tngq"
Oct 30 15:47:24.872080 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.871990    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3783d21c-aa91-4c4c-abe5-402f6e03da90-tls-cert\") pod \"authorino-6ddd779586-6tngq\" (UID: \"3783d21c-aa91-4c4c-abe5-402f6e03da90\") " pod="kuadrant-system/authorino-6ddd779586-6tngq"
Oct 30 15:47:24.874000 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.873967    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cad47cc-741c-4577-928f-00a391988f06-kube-api-access-z48h4" pod "5cad47cc-741c-4577-928f-00a391988f06" (UID: "5cad47cc-741c-4577-928f-00a391988f06"). InnerVolumeSpecName "kube-api-access-z48h4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:47:24.874106 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.873996    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffebfd77-3636-477f-97e7-50f8ee46f040-tls-cert" pod "ffebfd77-3636-477f-97e7-50f8ee46f040" (UID: "ffebfd77-3636-477f-97e7-50f8ee46f040"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:47:24.874521 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.874500    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3783d21c-aa91-4c4c-abe5-402f6e03da90-tls-cert\") pod \"authorino-6ddd779586-6tngq\" (UID: \"3783d21c-aa91-4c4c-abe5-402f6e03da90\") " pod="kuadrant-system/authorino-6ddd779586-6tngq"
Oct 30 15:47:24.895645 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.895614    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn8rj\" (UniqueName: \"kubernetes.io/projected/3783d21c-aa91-4c4c-abe5-402f6e03da90-kube-api-access-zn8rj\") pod \"authorino-6ddd779586-6tngq\" (UID: \"3783d21c-aa91-4c4c-abe5-402f6e03da90\") " pod="kuadrant-system/authorino-6ddd779586-6tngq"
Oct 30 15:47:24.964105 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.964068    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddd779586-6tngq"
Oct 30 15:47:24.972681 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.972660    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffebfd77-3636-477f-97e7-50f8ee46f040-kube-api-access-rm2d6\" (UniqueName: \"kubernetes.io/projected/ffebfd77-3636-477f-97e7-50f8ee46f040-kube-api-access-rm2d6\") pod \"ffebfd77-3636-477f-97e7-50f8ee46f040\" (UID: \"ffebfd77-3636-477f-97e7-50f8ee46f040\") "
Oct 30 15:47:24.972844 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.972829    4585 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ffebfd77-3636-477f-97e7-50f8ee46f040-tls-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:47:24.972945 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.972850    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z48h4\" (UniqueName: \"kubernetes.io/projected/5cad47cc-741c-4577-928f-00a391988f06-kube-api-access-z48h4\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:47:24.974619 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:24.974589    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffebfd77-3636-477f-97e7-50f8ee46f040-kube-api-access-rm2d6" pod "ffebfd77-3636-477f-97e7-50f8ee46f040" (UID: "ffebfd77-3636-477f-97e7-50f8ee46f040"). InnerVolumeSpecName "kube-api-access-rm2d6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:47:25.073785 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:25.073755    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm2d6\" (UniqueName: \"kubernetes.io/projected/ffebfd77-3636-477f-97e7-50f8ee46f040-kube-api-access-rm2d6\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:47:25.259561 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:25.259484    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6ddd779586-6tngq"]
Oct 30 15:47:25.262421 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:47:25.262389    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3783d21c_aa91_4c4c_abe5_402f6e03da90.slice/crio-ffeee789e4c9a7d33024f107824f282ff9f45595af5ade57bf1d695c4db715a9 WatchSource:0}: Error finding container ffeee789e4c9a7d33024f107824f282ff9f45595af5ade57bf1d695c4db715a9: Status 404 returned error can't find the container with id ffeee789e4c9a7d33024f107824f282ff9f45595af5ade57bf1d695c4db715a9
Oct 30 15:47:25.721455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:25.721418    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddd779586-6tngq" event={"ID":"3783d21c-aa91-4c4c-abe5-402f6e03da90","Type":"ContainerStarted","Data":"ffeee789e4c9a7d33024f107824f282ff9f45595af5ade57bf1d695c4db715a9"}
Oct 30 15:47:25.721455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:25.721441    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-r6xxv"
Oct 30 15:47:25.721655 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:25.721442    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-szsfk"
Oct 30 15:47:25.796471 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:25.796383    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-r6xxv"]
Oct 30 15:47:25.800633 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:25.800608    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-r6xxv"]
Oct 30 15:47:25.859557 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:25.859522    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-szsfk"]
Oct 30 15:47:25.863590 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:25.863561    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-szsfk"]
Oct 30 15:47:26.699665 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:26.699634    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5cad47cc-741c-4577-928f-00a391988f06" path="/var/lib/kubelet/pods/5cad47cc-741c-4577-928f-00a391988f06/volumes"
Oct 30 15:47:26.700043 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:26.699896    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffebfd77-3636-477f-97e7-50f8ee46f040" path="/var/lib/kubelet/pods/ffebfd77-3636-477f-97e7-50f8ee46f040/volumes"
Oct 30 15:47:26.727096 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:26.727059    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddd779586-6tngq" event={"ID":"3783d21c-aa91-4c4c-abe5-402f6e03da90","Type":"ContainerStarted","Data":"4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250"}
Oct 30 15:47:26.767722 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:26.767666    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6ddd779586-6tngq" podStartSLOduration=2.349024274 podStartE2EDuration="2.767649556s" totalImagesPullingTime="418.625282ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:47:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:47:26.766597739 +0000 UTC m=+556.202202087" watchObservedRunningTime="2026-10-30 15:47:26.767649556 +0000 UTC m=+556.203253903"
Oct 30 15:47:26.793404 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:26.793369    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wsvw2"]
Oct 30 15:47:26.793574 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:26.793554    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2" podUID="16e5eca1-86b2-4629-8ea0-f0bec633f314" containerName="authorino" containerID="cri-o://2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4" gracePeriod=30
Oct 30 15:47:27.048063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.048036    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2"
Oct 30 15:47:27.190214 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.190169    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16e5eca1-86b2-4629-8ea0-f0bec633f314-kube-api-access-jfm6z\" (UniqueName: \"kubernetes.io/projected/16e5eca1-86b2-4629-8ea0-f0bec633f314-kube-api-access-jfm6z\") pod \"16e5eca1-86b2-4629-8ea0-f0bec633f314\" (UID: \"16e5eca1-86b2-4629-8ea0-f0bec633f314\") "
Oct 30 15:47:27.196232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.196190    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16e5eca1-86b2-4629-8ea0-f0bec633f314-kube-api-access-jfm6z" pod "16e5eca1-86b2-4629-8ea0-f0bec633f314" (UID: "16e5eca1-86b2-4629-8ea0-f0bec633f314"). InnerVolumeSpecName "kube-api-access-jfm6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:47:27.291693 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.291612    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfm6z\" (UniqueName: \"kubernetes.io/projected/16e5eca1-86b2-4629-8ea0-f0bec633f314-kube-api-access-jfm6z\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:47:27.732518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.732476    4585 generic.go:356] "Generic (PLEG): container finished" podID="16e5eca1-86b2-4629-8ea0-f0bec633f314" containerID="2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4" exitCode=0
Oct 30 15:47:27.732999 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.732535    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2"
Oct 30 15:47:27.732999 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.732560    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2" event={"ID":"16e5eca1-86b2-4629-8ea0-f0bec633f314","Type":"ContainerDied","Data":"2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4"}
Oct 30 15:47:27.732999 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.732603    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-wsvw2" event={"ID":"16e5eca1-86b2-4629-8ea0-f0bec633f314","Type":"ContainerDied","Data":"cd822e447fa679371c3e56b61410db81962f6e4344e6f03c220f2600afcf6058"}
Oct 30 15:47:27.732999 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.732618    4585 scope.go:122] "RemoveContainer" containerID="2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4"
Oct 30 15:47:27.741883 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.741860    4585 scope.go:122] "RemoveContainer" containerID="2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4"
Oct 30 15:47:27.742174 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:47:27.742154    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4\": container with ID starting with 2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4 not found: ID does not exist" containerID="2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4"
Oct 30 15:47:27.742242 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.742186    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4"} err="failed to get container status \"2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4\": rpc error: code = NotFound desc = could not find container \"2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4\": container with ID starting with 2ec79d2415ba30e883d9f5de0e3847e80008a4414bdcfde00712b53a2c77d7a4 not found: ID does not exist"
Oct 30 15:47:27.791323 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.791294    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wsvw2"]
Oct 30 15:47:27.803397 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:27.803368    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-wsvw2"]
Oct 30 15:47:28.699359 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:47:28.699324    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16e5eca1-86b2-4629-8ea0-f0bec633f314" path="/var/lib/kubelet/pods/16e5eca1-86b2-4629-8ea0-f0bec633f314/volumes"
Oct 30 15:47:47.960244 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:47:47.960205    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:48:10.739587 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:48:10.739556    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:48:10.741541 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:48:10.741512    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:48:17.961335 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:48:17.961285    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:48:47.960895 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:48:47.960856    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:49:17.481010 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.480973    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"]
Oct 30 15:49:17.482265 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.482236    4585 cpu_manager.go:414] "containerMap: removing container" podUID="16e5eca1-86b2-4629-8ea0-f0bec633f314" containerName="authorino"
Oct 30 15:49:17.482606 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.482585    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16e5eca1-86b2-4629-8ea0-f0bec633f314" containerName="authorino"
Oct 30 15:49:17.482918 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.482904    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="16e5eca1-86b2-4629-8ea0-f0bec633f314" containerName="authorino"
Oct 30 15:49:17.487149 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.487096    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.507301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.507270    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 15:49:17.509201 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.509174    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"]
Oct 30 15:49:17.519682 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.519657    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 15:49:17.519865 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.519731    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 15:49:17.519979 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.519914    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kvt8s\""
Oct 30 15:49:17.600986 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.600951    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.600986 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.600989    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.601212 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.601014    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjpgd\" (UniqueName: \"kubernetes.io/projected/a63940b1-9bb4-4a37-bd76-c560b72869ee-kube-api-access-rjpgd\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.601212 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.601156    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a63940b1-9bb4-4a37-bd76-c560b72869ee-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.601212 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.601207    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.601318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.601228    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.701819 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.701788    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.701819 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.701823    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.702076 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.701846    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjpgd\" (UniqueName: \"kubernetes.io/projected/a63940b1-9bb4-4a37-bd76-c560b72869ee-kube-api-access-rjpgd\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.702076 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.701876    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a63940b1-9bb4-4a37-bd76-c560b72869ee-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.702076 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.701912    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.702076 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.701929    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.702299 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.702273    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.702395 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.702342    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.702527 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.702489    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.704286 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.704266    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a63940b1-9bb4-4a37-bd76-c560b72869ee-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.704678 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.704658    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a63940b1-9bb4-4a37-bd76-c560b72869ee-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.725197 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.725168    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjpgd\" (UniqueName: \"kubernetes.io/projected/a63940b1-9bb4-4a37-bd76-c560b72869ee-kube-api-access-rjpgd\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6\" (UID: \"a63940b1-9bb4-4a37-bd76-c560b72869ee\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.806322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:17.806236    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:17.962073 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:49:17.962030    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:49:18.106364 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:18.106320    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"]
Oct 30 15:49:18.107462 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:49:18.107436    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63940b1_9bb4_4a37_bd76_c560b72869ee.slice/crio-5c33956bb9a400ba93336261c0a1db93cd6a3f0ee573317c96f1749a557968cc WatchSource:0}: Error finding container 5c33956bb9a400ba93336261c0a1db93cd6a3f0ee573317c96f1749a557968cc: Status 404 returned error can't find the container with id 5c33956bb9a400ba93336261c0a1db93cd6a3f0ee573317c96f1749a557968cc
Oct 30 15:49:18.108735 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:18.108717    4585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 15:49:18.164288 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:18.164250    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6" event={"ID":"a63940b1-9bb4-4a37-bd76-c560b72869ee","Type":"ContainerStarted","Data":"5c33956bb9a400ba93336261c0a1db93cd6a3f0ee573317c96f1749a557968cc"}
Oct 30 15:49:19.267932 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.266564    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"]
Oct 30 15:49:19.270831 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.270242    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.290478 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.290271    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"]
Oct 30 15:49:19.301023 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.300992    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 30 15:49:19.417229 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.417196    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.417377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.417234    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.417377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.417303    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmdhw\" (UniqueName: \"kubernetes.io/projected/f1d46283-c4f5-4114-a114-44900ef4eb18-kube-api-access-dmdhw\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.417377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.417356    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.417578 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.417440    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d46283-c4f5-4114-a114-44900ef4eb18-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.417578 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.417546    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.518536 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.518461    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.518536 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.518502    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d46283-c4f5-4114-a114-44900ef4eb18-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.518536 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.518524    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.518801 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.518576    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.518801 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.518594    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.518801 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.518711    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmdhw\" (UniqueName: \"kubernetes.io/projected/f1d46283-c4f5-4114-a114-44900ef4eb18-kube-api-access-dmdhw\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.519019 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.518960    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.519019 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.518986    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.519440 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.519035    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.521019 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.520995    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1d46283-c4f5-4114-a114-44900ef4eb18-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.521276 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.521256    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1d46283-c4f5-4114-a114-44900ef4eb18-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.543308 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.543272    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmdhw\" (UniqueName: \"kubernetes.io/projected/f1d46283-c4f5-4114-a114-44900ef4eb18-kube-api-access-dmdhw\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hwb9z\" (UID: \"f1d46283-c4f5-4114-a114-44900ef4eb18\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.593139 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.593105    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:19.894445 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:19.894330    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"]
Oct 30 15:49:19.895478 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:49:19.895449    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d46283_c4f5_4114_a114_44900ef4eb18.slice/crio-0b392eff0939afd2921dafe2c600affea30fd55a40396f29b3f4d65c2111271f WatchSource:0}: Error finding container 0b392eff0939afd2921dafe2c600affea30fd55a40396f29b3f4d65c2111271f: Status 404 returned error can't find the container with id 0b392eff0939afd2921dafe2c600affea30fd55a40396f29b3f4d65c2111271f
Oct 30 15:49:20.174360 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:20.174318    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6" event={"ID":"a63940b1-9bb4-4a37-bd76-c560b72869ee","Type":"ContainerStarted","Data":"d3a2d9d8e4078656d10fff2a2609420585a7c78e8a6858a6dbac26b5d79b5466"}
Oct 30 15:49:20.175838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:20.175813    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z" event={"ID":"f1d46283-c4f5-4114-a114-44900ef4eb18","Type":"ContainerStarted","Data":"681f11dd56939f15da1ad820f41b9acbd6400064500fc78008effbe5800509cf"}
Oct 30 15:49:20.175942 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:20.175856    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z" event={"ID":"f1d46283-c4f5-4114-a114-44900ef4eb18","Type":"ContainerStarted","Data":"0b392eff0939afd2921dafe2c600affea30fd55a40396f29b3f4d65c2111271f"}
Oct 30 15:49:20.217154 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:20.217100    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6" podStartSLOduration=1.978415829 podStartE2EDuration="3.217084035s" totalImagesPullingTime="1.238668206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:49:20.216899045 +0000 UTC m=+669.652503393" watchObservedRunningTime="2026-10-30 15:49:20.217084035 +0000 UTC m=+669.652688385"
Oct 30 15:49:20.259752 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:20.259703    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z" podStartSLOduration=1.060553941 podStartE2EDuration="1.259690464s" totalImagesPullingTime="199.136523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:49:20.258745105 +0000 UTC m=+669.694349454" watchObservedRunningTime="2026-10-30 15:49:20.259690464 +0000 UTC m=+669.695294812"
Oct 30 15:49:20.593515 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:20.593431    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:20.806681 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:20.806644    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:22.562810 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.562769    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"]
Oct 30 15:49:22.566720 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.566690    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.594391 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.594325    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"]
Oct 30 15:49:22.596422 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.596401    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 15:49:22.664658 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.664270    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnj4s\" (UniqueName: \"kubernetes.io/projected/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-kube-api-access-pnj4s\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.664658 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.664385    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.664658 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.664422    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.664658 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.664495    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.664658 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.664530    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.664658 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.664590    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.765593 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.765558    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnj4s\" (UniqueName: \"kubernetes.io/projected/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-kube-api-access-pnj4s\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.765775 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.765604    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.765775 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.765628    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.765775 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.765661    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.765775 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.765692    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.766009 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.765857    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.766080 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.766059    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.766144 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.766112    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.766341 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.766310    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.767944 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.767917    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.768203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.768184    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.788938 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.788908    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnj4s\" (UniqueName: \"kubernetes.io/projected/f2bc9d3b-4eb9-446f-98c2-1527620c7c05-kube-api-access-pnj4s\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-hxckg\" (UID: \"f2bc9d3b-4eb9-446f-98c2-1527620c7c05\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:22.892956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:22.892867    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:23.196018 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:23.195990    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"]
Oct 30 15:49:23.197709 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:49:23.197687    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2bc9d3b_4eb9_446f_98c2_1527620c7c05.slice/crio-378e5ef0a678adacaf159102f4ab1ac3ff451c2917a5588688c9255fd3b1ba0f WatchSource:0}: Error finding container 378e5ef0a678adacaf159102f4ab1ac3ff451c2917a5588688c9255fd3b1ba0f: Status 404 returned error can't find the container with id 378e5ef0a678adacaf159102f4ab1ac3ff451c2917a5588688c9255fd3b1ba0f
Oct 30 15:49:24.215424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:24.215109    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg" event={"ID":"f2bc9d3b-4eb9-446f-98c2-1527620c7c05","Type":"ContainerStarted","Data":"21ce63f8a0d967595e904dfb1fc129c26a46286f17989e1211cba79faad14174"}
Oct 30 15:49:24.215424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:24.215217    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg" event={"ID":"f2bc9d3b-4eb9-446f-98c2-1527620c7c05","Type":"ContainerStarted","Data":"378e5ef0a678adacaf159102f4ab1ac3ff451c2917a5588688c9255fd3b1ba0f"}
Oct 30 15:49:24.258363 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:24.258295    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg" podStartSLOduration=2.064764268 podStartE2EDuration="2.258278355s" totalImagesPullingTime="193.514087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:49:24.257456186 +0000 UTC m=+673.693060535" watchObservedRunningTime="2026-10-30 15:49:24.258278355 +0000 UTC m=+673.693882704"
Oct 30 15:49:24.893294 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:24.893257    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:26.649799 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.649766    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"]
Oct 30 15:49:26.653266 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.653238    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.673749 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.673720    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Oct 30 15:49:26.686824 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.686794    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"]
Oct 30 15:49:26.708060 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.708026    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.708214 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.708077    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.708214 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.708149    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.708214 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.708194    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvbsb\" (UniqueName: \"kubernetes.io/projected/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-kube-api-access-vvbsb\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.708337 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.708316    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.708405 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.708377    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.809317 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.809272    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.809500 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.809342    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.809500 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.809391    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.809500 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.809418    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvbsb\" (UniqueName: \"kubernetes.io/projected/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-kube-api-access-vvbsb\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.809500 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.809495    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.809738 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.809545    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.809908 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.809878    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.810122 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.810095    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.810195 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.810136    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.811802 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.811771    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.812134 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.812111    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.833143 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.833113    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvbsb\" (UniqueName: \"kubernetes.io/projected/b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d-kube-api-access-vvbsb\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf\" (UID: \"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:26.968553 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:26.968452    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:27.286449 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:27.282643    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"]
Oct 30 15:49:27.806691 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:27.806612    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:27.819864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:27.819832    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:28.198198 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.198151    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"]
Oct 30 15:49:28.201856 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.201829    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.220109 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.220080    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 30 15:49:28.224061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.223701    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.224061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.223750    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.224061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.223786    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.224061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.223831    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgshw\" (UniqueName: \"kubernetes.io/projected/84e32e1d-bae5-4233-ac38-50ff6ccaa774-kube-api-access-jgshw\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.224061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.223900    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84e32e1d-bae5-4233-ac38-50ff6ccaa774-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.224061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.223922    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.225281 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.225247    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"]
Oct 30 15:49:28.237420 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.237384    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf" event={"ID":"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d","Type":"ContainerStarted","Data":"f2b39ff996145752dd1e9a26901bba1dd474c294d247970a9bf210af246b0419"}
Oct 30 15:49:28.237583 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.237452    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf" event={"ID":"b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d","Type":"ContainerStarted","Data":"f0d5de111fea49bd615dfb10af303792255ef83d2afd57fec37246f4e49a1581"}
Oct 30 15:49:28.249431 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.249403    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6"
Oct 30 15:49:28.310368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.307545    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf" podStartSLOduration=2.079274034 podStartE2EDuration="2.307525994s" totalImagesPullingTime="228.25196ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:49:28.305861631 +0000 UTC m=+677.741465981" watchObservedRunningTime="2026-10-30 15:49:28.307525994 +0000 UTC m=+677.743130344"
Oct 30 15:49:28.327832 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.327791    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84e32e1d-bae5-4233-ac38-50ff6ccaa774-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.328024 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.327839    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.328024 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.327974    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.328024 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.328003    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.328191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.328039    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.328191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.328086    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgshw\" (UniqueName: \"kubernetes.io/projected/84e32e1d-bae5-4233-ac38-50ff6ccaa774-kube-api-access-jgshw\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.329320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.329289    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.329847 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.329790    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.330086 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.330053    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.331815 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.331794    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84e32e1d-bae5-4233-ac38-50ff6ccaa774-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.333020 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.333000    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84e32e1d-bae5-4233-ac38-50ff6ccaa774-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.352089 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.352051    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgshw\" (UniqueName: \"kubernetes.io/projected/84e32e1d-bae5-4233-ac38-50ff6ccaa774-kube-api-access-jgshw\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq\" (UID: \"84e32e1d-bae5-4233-ac38-50ff6ccaa774\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.516194 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.516111    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:28.812487 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.812461    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"]
Oct 30 15:49:28.813992 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:49:28.813962    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84e32e1d_bae5_4233_ac38_50ff6ccaa774.slice/crio-7f2f1793d259f99976ee1c7b514c6c5cb3b5d4595d50ceeaf85d6d2e5fcaea06 WatchSource:0}: Error finding container 7f2f1793d259f99976ee1c7b514c6c5cb3b5d4595d50ceeaf85d6d2e5fcaea06: Status 404 returned error can't find the container with id 7f2f1793d259f99976ee1c7b514c6c5cb3b5d4595d50ceeaf85d6d2e5fcaea06
Oct 30 15:49:28.969315 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:28.969279    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:29.243301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:29.243253    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" event={"ID":"84e32e1d-bae5-4233-ac38-50ff6ccaa774","Type":"ContainerStarted","Data":"9f3de49931100e47d4a06b62f96e1a2e858084183d3dd34f8b141289c7be6af2"}
Oct 30 15:49:29.243301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:29.243305    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" event={"ID":"84e32e1d-bae5-4233-ac38-50ff6ccaa774","Type":"ContainerStarted","Data":"7f2f1793d259f99976ee1c7b514c6c5cb3b5d4595d50ceeaf85d6d2e5fcaea06"}
Oct 30 15:49:29.285272 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:29.285224    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podStartSLOduration=1.078786483 podStartE2EDuration="1.285209197s" totalImagesPullingTime="206.422714ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:49:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:49:29.284539916 +0000 UTC m=+678.720144263" watchObservedRunningTime="2026-10-30 15:49:29.285209197 +0000 UTC m=+678.720813535"
Oct 30 15:49:29.516619 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:29.516577    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:29.594115 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:29.594083    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:29.606799 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:29.606769    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:30.262846 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:30.262814    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hwb9z"
Oct 30 15:49:32.894049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:32.894013    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:32.909540 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:32.909510    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:33.274571 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:33.274495    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-hxckg"
Oct 30 15:49:36.969193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:36.969153    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:36.981511 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:36.981483    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:37.293650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:37.293566    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf"
Oct 30 15:49:38.516552 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:38.516519    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:38.528980 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:38.528953    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:39.301691 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:39.301666    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:49:42.281163 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.281131    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65cff8b9d5-r6lh9"]
Oct 30 15:49:42.284660 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.284629    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65cff8b9d5-r6lh9"
Oct 30 15:49:42.305085 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.305062    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65cff8b9d5-r6lh9"]
Oct 30 15:49:42.349838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.349804    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f874c784-9b43-41c6-bdaf-b029d9ca989d-tls-cert\") pod \"authorino-65cff8b9d5-r6lh9\" (UID: \"f874c784-9b43-41c6-bdaf-b029d9ca989d\") " pod="kuadrant-system/authorino-65cff8b9d5-r6lh9"
Oct 30 15:49:42.350027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.349871    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldj22\" (UniqueName: \"kubernetes.io/projected/f874c784-9b43-41c6-bdaf-b029d9ca989d-kube-api-access-ldj22\") pod \"authorino-65cff8b9d5-r6lh9\" (UID: \"f874c784-9b43-41c6-bdaf-b029d9ca989d\") " pod="kuadrant-system/authorino-65cff8b9d5-r6lh9"
Oct 30 15:49:42.451038 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.450999    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f874c784-9b43-41c6-bdaf-b029d9ca989d-tls-cert\") pod \"authorino-65cff8b9d5-r6lh9\" (UID: \"f874c784-9b43-41c6-bdaf-b029d9ca989d\") " pod="kuadrant-system/authorino-65cff8b9d5-r6lh9"
Oct 30 15:49:42.451221 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.451055    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldj22\" (UniqueName: \"kubernetes.io/projected/f874c784-9b43-41c6-bdaf-b029d9ca989d-kube-api-access-ldj22\") pod \"authorino-65cff8b9d5-r6lh9\" (UID: \"f874c784-9b43-41c6-bdaf-b029d9ca989d\") " pod="kuadrant-system/authorino-65cff8b9d5-r6lh9"
Oct 30 15:49:42.453492 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.453467    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f874c784-9b43-41c6-bdaf-b029d9ca989d-tls-cert\") pod \"authorino-65cff8b9d5-r6lh9\" (UID: \"f874c784-9b43-41c6-bdaf-b029d9ca989d\") " pod="kuadrant-system/authorino-65cff8b9d5-r6lh9"
Oct 30 15:49:42.475088 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.475053    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldj22\" (UniqueName: \"kubernetes.io/projected/f874c784-9b43-41c6-bdaf-b029d9ca989d-kube-api-access-ldj22\") pod \"authorino-65cff8b9d5-r6lh9\" (UID: \"f874c784-9b43-41c6-bdaf-b029d9ca989d\") " pod="kuadrant-system/authorino-65cff8b9d5-r6lh9"
Oct 30 15:49:42.597087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.597005    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65cff8b9d5-r6lh9"
Oct 30 15:49:42.900127 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:42.900095    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65cff8b9d5-r6lh9"]
Oct 30 15:49:42.901165 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:49:42.901140    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf874c784_9b43_41c6_bdaf_b029d9ca989d.slice/crio-6502c4e1e9d62422102dc07826f373c45fdc1ab9573bc5151b42b9434beca556 WatchSource:0}: Error finding container 6502c4e1e9d62422102dc07826f373c45fdc1ab9573bc5151b42b9434beca556: Status 404 returned error can't find the container with id 6502c4e1e9d62422102dc07826f373c45fdc1ab9573bc5151b42b9434beca556
Oct 30 15:49:43.308674 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:43.308592    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65cff8b9d5-r6lh9" event={"ID":"f874c784-9b43-41c6-bdaf-b029d9ca989d","Type":"ContainerStarted","Data":"6502c4e1e9d62422102dc07826f373c45fdc1ab9573bc5151b42b9434beca556"}
Oct 30 15:49:44.315455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.315413    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65cff8b9d5-r6lh9" event={"ID":"f874c784-9b43-41c6-bdaf-b029d9ca989d","Type":"ContainerStarted","Data":"ae29323dfcaff3f6114cebc0f68deca5f03f088a6abc864273c346769a724efc"}
Oct 30 15:49:44.359157 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.359092    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-65cff8b9d5-r6lh9" podStartSLOduration=1.485417484 podStartE2EDuration="2.359076633s" totalImagesPullingTime="873.659149ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:49:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:49:44.357226233 +0000 UTC m=+693.792830592" watchObservedRunningTime="2026-10-30 15:49:44.359076633 +0000 UTC m=+693.794680980"
Oct 30 15:49:44.385999 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.385964    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6ddd779586-6tngq"]
Oct 30 15:49:44.386228 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.386197    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6ddd779586-6tngq" podUID="3783d21c-aa91-4c4c-abe5-402f6e03da90" containerName="authorino" containerID="cri-o://4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250" gracePeriod=30
Oct 30 15:49:44.657413 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.657379    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddd779586-6tngq"
Oct 30 15:49:44.777364 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.777313    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3783d21c-aa91-4c4c-abe5-402f6e03da90-kube-api-access-zn8rj\" (UniqueName: \"kubernetes.io/projected/3783d21c-aa91-4c4c-abe5-402f6e03da90-kube-api-access-zn8rj\") pod \"3783d21c-aa91-4c4c-abe5-402f6e03da90\" (UID: \"3783d21c-aa91-4c4c-abe5-402f6e03da90\") "
Oct 30 15:49:44.777553 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.777404    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3783d21c-aa91-4c4c-abe5-402f6e03da90-tls-cert\" (UniqueName: \"kubernetes.io/secret/3783d21c-aa91-4c4c-abe5-402f6e03da90-tls-cert\") pod \"3783d21c-aa91-4c4c-abe5-402f6e03da90\" (UID: \"3783d21c-aa91-4c4c-abe5-402f6e03da90\") "
Oct 30 15:49:44.779647 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.779608    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3783d21c-aa91-4c4c-abe5-402f6e03da90-kube-api-access-zn8rj" pod "3783d21c-aa91-4c4c-abe5-402f6e03da90" (UID: "3783d21c-aa91-4c4c-abe5-402f6e03da90"). InnerVolumeSpecName "kube-api-access-zn8rj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:49:44.787914 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.787888    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3783d21c-aa91-4c4c-abe5-402f6e03da90-tls-cert" pod "3783d21c-aa91-4c4c-abe5-402f6e03da90" (UID: "3783d21c-aa91-4c4c-abe5-402f6e03da90"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:49:44.879379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.879333    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn8rj\" (UniqueName: \"kubernetes.io/projected/3783d21c-aa91-4c4c-abe5-402f6e03da90-kube-api-access-zn8rj\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:49:44.879379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:44.879376    4585 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3783d21c-aa91-4c4c-abe5-402f6e03da90-tls-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:49:45.320995 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:45.320961    4585 generic.go:356] "Generic (PLEG): container finished" podID="3783d21c-aa91-4c4c-abe5-402f6e03da90" containerID="4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250" exitCode=0
Oct 30 15:49:45.321455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:45.321004    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddd779586-6tngq"
Oct 30 15:49:45.321455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:45.321050    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddd779586-6tngq" event={"ID":"3783d21c-aa91-4c4c-abe5-402f6e03da90","Type":"ContainerDied","Data":"4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250"}
Oct 30 15:49:45.321455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:45.321092    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddd779586-6tngq" event={"ID":"3783d21c-aa91-4c4c-abe5-402f6e03da90","Type":"ContainerDied","Data":"ffeee789e4c9a7d33024f107824f282ff9f45595af5ade57bf1d695c4db715a9"}
Oct 30 15:49:45.321455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:45.321115    4585 scope.go:122] "RemoveContainer" containerID="4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250"
Oct 30 15:49:45.331544 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:45.331526    4585 scope.go:122] "RemoveContainer" containerID="4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250"
Oct 30 15:49:45.331829 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:49:45.331807    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250\": container with ID starting with 4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250 not found: ID does not exist" containerID="4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250"
Oct 30 15:49:45.331876 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:45.331839    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250"} err="failed to get container status \"4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250\": rpc error: code = NotFound desc = could not find container \"4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250\": container with ID starting with 4449c4c5dfd8377ec0daffa09a9326ff86d034d46707fa31aaf2ae95e3fe0250 not found: ID does not exist"
Oct 30 15:49:45.371134 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:45.371104    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6ddd779586-6tngq"]
Oct 30 15:49:45.376892 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:45.376870    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6ddd779586-6tngq"]
Oct 30 15:49:46.711787 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:46.711336    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3783d21c-aa91-4c4c-abe5-402f6e03da90" path="/var/lib/kubelet/pods/3783d21c-aa91-4c4c-abe5-402f6e03da90/volumes"
Oct 30 15:49:47.961058 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:49:47.961014    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:49:51.972874 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:51.972839    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"]
Oct 30 15:49:51.973523 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:51.973490    4585 cpu_manager.go:414] "containerMap: removing container" podUID="3783d21c-aa91-4c4c-abe5-402f6e03da90" containerName="authorino"
Oct 30 15:49:51.973523 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:51.973511    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3783d21c-aa91-4c4c-abe5-402f6e03da90" containerName="authorino"
Oct 30 15:49:51.973650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:51.973606    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="3783d21c-aa91-4c4c-abe5-402f6e03da90" containerName="authorino"
Oct 30 15:49:51.978392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:51.978369    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:51.997501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:51.997469    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 15:49:51.997651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:51.997507    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-x4s56\""
Oct 30 15:49:52.009958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.009929    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"]
Oct 30 15:49:52.049194 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.049155    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.049409 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.049216    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.049409 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.049291    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxnp4\" (UniqueName: \"kubernetes.io/projected/f6fa0c9c-0a86-4a03-8af0-561d339e6831-kube-api-access-cxnp4\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.049409 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.049338    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.049598 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.049483    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.049598 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.049519    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.049598 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.049548    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.049756 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.049641    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.049756 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.049685    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151036 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.150994    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151036 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151045    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151076    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151131    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151155    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxnp4\" (UniqueName: \"kubernetes.io/projected/f6fa0c9c-0a86-4a03-8af0-561d339e6831-kube-api-access-cxnp4\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151187    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151232    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151255    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151284    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151724 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151593    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151724 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151667    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151839 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151768    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151839 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151827    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.151917 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.151901    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.153565 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.153543    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.153836 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.153813    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.178243 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.178214    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6fa0c9c-0a86-4a03-8af0-561d339e6831-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.178551 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.178529    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxnp4\" (UniqueName: \"kubernetes.io/projected/f6fa0c9c-0a86-4a03-8af0-561d339e6831-kube-api-access-cxnp4\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-fh66k\" (UID: \"f6fa0c9c-0a86-4a03-8af0-561d339e6831\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.290685 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.290588    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:52.604326 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:52.604298    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"]
Oct 30 15:49:52.605161 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:49:52.605130    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6fa0c9c_0a86_4a03_8af0_561d339e6831.slice/crio-ce16d892c902688cc2b07651cef5120b9115a838826de9fed3c1823db795ac37 WatchSource:0}: Error finding container ce16d892c902688cc2b07651cef5120b9115a838826de9fed3c1823db795ac37: Status 404 returned error can't find the container with id ce16d892c902688cc2b07651cef5120b9115a838826de9fed3c1823db795ac37
Oct 30 15:49:53.357268 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:53.357236    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k" event={"ID":"f6fa0c9c-0a86-4a03-8af0-561d339e6831","Type":"ContainerStarted","Data":"ce16d892c902688cc2b07651cef5120b9115a838826de9fed3c1823db795ac37"}
Oct 30 15:49:55.362373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:55.362326    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:49:55.362644 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:55.362408    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:49:55.362644 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:55.362441    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:49:56.372195 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:56.372157    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k" event={"ID":"f6fa0c9c-0a86-4a03-8af0-561d339e6831","Type":"ContainerStarted","Data":"d3e81a6c8f6f6553507e16c2fc46f767ad1381b579aecc36a721b823722cf06b"}
Oct 30 15:49:56.450145 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:56.450079    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k" podStartSLOduration=2.695651709 podStartE2EDuration="5.450058041s" totalImagesPullingTime="2.754406332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:49:56.424414874 +0000 UTC m=+705.860019238" watchObservedRunningTime="2026-10-30 15:49:56.450058041 +0000 UTC m=+705.885662391"
Oct 30 15:49:57.291024 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:57.290990    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:49:57.292437 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:57.292406    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fh66k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 15:49:57.292574 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:57.292461    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k" podUID="f6fa0c9c-0a86-4a03-8af0-561d339e6831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 15:49:58.291049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:58.291009    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fh66k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 15:49:58.291499 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:58.291062    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k" podUID="f6fa0c9c-0a86-4a03-8af0-561d339e6831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 15:49:59.291016 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:59.290980    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fh66k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 15:49:59.291172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:49:59.291040    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k" podUID="f6fa0c9c-0a86-4a03-8af0-561d339e6831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 15:50:00.291415 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:00.291376    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fh66k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 15:50:00.291846 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:00.291438    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k" podUID="f6fa0c9c-0a86-4a03-8af0-561d339e6831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 15:50:01.291693 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:01.291656    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fh66k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 15:50:01.292119 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:01.291726    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k" podUID="f6fa0c9c-0a86-4a03-8af0-561d339e6831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 15:50:02.291383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:02.291328    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-fh66k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 15:50:02.291614 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:02.291400    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k" podUID="f6fa0c9c-0a86-4a03-8af0-561d339e6831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 15:50:03.656416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:03.656379    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:50:03.656929 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:03.656794    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:50:03.657512 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:03.657496    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-fh66k"
Oct 30 15:50:07.015530 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.015495    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"]
Oct 30 15:50:07.018618 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.018597    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.043197 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.043166    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"]
Oct 30 15:50:07.118215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.118180    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.118215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.118213    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4adedae1-f191-4075-8059-805e373e4418-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.118517 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.118250    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68bkm\" (UniqueName: \"kubernetes.io/projected/4adedae1-f191-4075-8059-805e373e4418-kube-api-access-68bkm\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.118517 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.118387    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.118517 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.118435    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4adedae1-f191-4075-8059-805e373e4418-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.118661 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.118524    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.118661 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.118551    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.118661 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.118598    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4adedae1-f191-4075-8059-805e373e4418-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.118757 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.118663    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219209 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219175    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4adedae1-f191-4075-8059-805e373e4418-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219426 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219224    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219426 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219244    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219426 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219270    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4adedae1-f191-4075-8059-805e373e4418-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219608 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219454    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219608 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219514    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219608 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219537    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4adedae1-f191-4075-8059-805e373e4418-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219761 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219606    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68bkm\" (UniqueName: \"kubernetes.io/projected/4adedae1-f191-4075-8059-805e373e4418-kube-api-access-68bkm\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219761 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219655    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219860 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219770    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219860 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219822    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.219987 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219963    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.220045 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.219999    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.220124 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.220103    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4adedae1-f191-4075-8059-805e373e4418-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.221692 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.221672    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4adedae1-f191-4075-8059-805e373e4418-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.222014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.221991    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4adedae1-f191-4075-8059-805e373e4418-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.243043 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.243015    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68bkm\" (UniqueName: \"kubernetes.io/projected/4adedae1-f191-4075-8059-805e373e4418-kube-api-access-68bkm\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.243162 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.243042    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4adedae1-f191-4075-8059-805e373e4418-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-l7x5w\" (UID: \"4adedae1-f191-4075-8059-805e373e4418\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.331596 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.331503    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:07.644815 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.644786    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"]
Oct 30 15:50:07.645859 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:50:07.645838    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4adedae1_f191_4075_8059_805e373e4418.slice/crio-491d0533cb985038860e7edcba0b9bc5ec646c802d2646abf2ab2cc725233c07 WatchSource:0}: Error finding container 491d0533cb985038860e7edcba0b9bc5ec646c802d2646abf2ab2cc725233c07: Status 404 returned error can't find the container with id 491d0533cb985038860e7edcba0b9bc5ec646c802d2646abf2ab2cc725233c07
Oct 30 15:50:07.648732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.648703    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:07.648803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.648764    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:07.648803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:07.648793    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:08.427528 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:08.427493    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w" event={"ID":"4adedae1-f191-4075-8059-805e373e4418","Type":"ContainerStarted","Data":"b00bd77fcce7fee47824ba5806237f7947bf1a4994146072732e690ff30a76a0"}
Oct 30 15:50:08.427931 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:08.427549    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w" event={"ID":"4adedae1-f191-4075-8059-805e373e4418","Type":"ContainerStarted","Data":"491d0533cb985038860e7edcba0b9bc5ec646c802d2646abf2ab2cc725233c07"}
Oct 30 15:50:08.489232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:08.489178    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w" podStartSLOduration=2.489159084 podStartE2EDuration="2.489159084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:08.488594748 +0000 UTC m=+717.924199096" watchObservedRunningTime="2026-10-30 15:50:08.489159084 +0000 UTC m=+717.924763432"
Oct 30 15:50:09.335546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:09.333696    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:09.354732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:09.354697    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 15:50:09.354907 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:09.354755    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w" podUID="4adedae1-f191-4075-8059-805e373e4418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 15:50:10.342380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:10.339754    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 15:50:10.342380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:10.339817    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w" podUID="4adedae1-f191-4075-8059-805e373e4418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 15:50:11.345830 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:11.345785    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 15:50:11.346473 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:11.346440    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w" podUID="4adedae1-f191-4075-8059-805e373e4418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 15:50:12.341374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:12.338733    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 15:50:12.341374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:12.338812    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w" podUID="4adedae1-f191-4075-8059-805e373e4418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 15:50:13.353559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:13.353516    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 15:50:13.354075 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:13.353624    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w" podUID="4adedae1-f191-4075-8059-805e373e4418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 15:50:14.332780 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:14.332743    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 15:50:14.332958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:14.332793    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w" podUID="4adedae1-f191-4075-8059-805e373e4418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 15:50:15.907720 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:15.907640    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:15.908215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:15.908184    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:15.909202 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:15.909181    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-l7x5w"
Oct 30 15:50:17.960713 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:50:17.960665    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:50:40.562756 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.562725    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"]
Oct 30 15:50:40.566358 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.566328    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.594056 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.594029    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"]
Oct 30 15:50:40.600512 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.600468    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-b63c3c-openshift-default-dockercfg-94q4v\""
Oct 30 15:50:40.660715 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.660677    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.660715 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.660711    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.660949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.660730    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.660949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.660751    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxjr5\" (UniqueName: \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-kube-api-access-xxjr5\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.660949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.660940    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.661093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.660969    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.661093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.661003    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.661093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.661031    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.661187 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.661109    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35c54ae6-a861-4efc-8398-78b4610e0c40-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.732597 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.732570    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"]
Oct 30 15:50:40.736283 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.736261    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.753500 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.753476    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-ba9db9-openshift-default-dockercfg-lczk5\""
Oct 30 15:50:40.761620 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.761597    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"]
Oct 30 15:50:40.762072 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762047    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35c54ae6-a861-4efc-8398-78b4610e0c40-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762096    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762116    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762143    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762244    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxjr5\" (UniqueName: \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-kube-api-access-xxjr5\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762361    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762391    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762440    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762464    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762662 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762567    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762813 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762791    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.762889 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.762866    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35c54ae6-a861-4efc-8398-78b4610e0c40-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.763158 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.763134    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.763264 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.763174    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.765203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.765183    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.765989 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.765960    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.803822 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.803784    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.804423 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.804403    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxjr5\" (UniqueName: \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-kube-api-access-xxjr5\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.863518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.863426    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.863518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.863489    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.863747 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.863720    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.863869 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.863843    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.864184 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.864159    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.864307 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.864289    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/876862ba-27c9-4c8b-b440-93ec2241b948-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.864441 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.864418    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/876862ba-27c9-4c8b-b440-93ec2241b948-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.864651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.864635    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.864700 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.864663    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mhnv\" (UniqueName: \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-kube-api-access-8mhnv\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.880089 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.880066    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:40.967183 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.967145    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.967384 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.967200    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.967384 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.967260    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.967384 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.967356    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/876862ba-27c9-4c8b-b440-93ec2241b948-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.967569 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.967387    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/876862ba-27c9-4c8b-b440-93ec2241b948-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.967569 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.967431    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.967569 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.967461    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mhnv\" (UniqueName: \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-kube-api-access-8mhnv\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.967569 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.967491    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.967569 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.967535    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.968095 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.968033    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.968568 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.968112    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.968568 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.968521    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.968568 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.968535    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.968568 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.968561    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/876862ba-27c9-4c8b-b440-93ec2241b948-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.970301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.970145    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.970646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.970620    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/876862ba-27c9-4c8b-b440-93ec2241b948-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.992546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.992518    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:40.992662 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:40.992608    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mhnv\" (UniqueName: \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-kube-api-access-8mhnv\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:41.048499 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.048464    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:41.209756 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:50:41.209721    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c54ae6_a861_4efc_8398_78b4610e0c40.slice/crio-bb4f9519ce66b5d91252db5815c1765ee988ab7e7dce8b32707806374f880317 WatchSource:0}: Error finding container bb4f9519ce66b5d91252db5815c1765ee988ab7e7dce8b32707806374f880317: Status 404 returned error can't find the container with id bb4f9519ce66b5d91252db5815c1765ee988ab7e7dce8b32707806374f880317
Oct 30 15:50:41.210265 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.210189    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"]
Oct 30 15:50:41.212076 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.212036    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:41.212169 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.212121    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:41.212216 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.212166    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:41.255745 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.255719    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"]
Oct 30 15:50:41.263012 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.262976    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.285840 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.285764    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"]
Oct 30 15:50:41.298700 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.298672    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-c129d1-openshift-default-dockercfg-qvk6s\""
Oct 30 15:50:41.367616 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.367585    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"]
Oct 30 15:50:41.369566 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:50:41.369542    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod876862ba_27c9_4c8b_b440_93ec2241b948.slice/crio-878e9eeb8399238701fd3e8e62f708c1d13edfe91579bf0bbd37cb8cda2efef0 WatchSource:0}: Error finding container 878e9eeb8399238701fd3e8e62f708c1d13edfe91579bf0bbd37cb8cda2efef0: Status 404 returned error can't find the container with id 878e9eeb8399238701fd3e8e62f708c1d13edfe91579bf0bbd37cb8cda2efef0
Oct 30 15:50:41.371620 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.371592    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.371744 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.371634    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bmlx\" (UniqueName: \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-kube-api-access-8bmlx\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.371887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.371799    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.371951 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.371935    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.372036 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.372018    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba4c3f49-4422-483f-b24f-41545ed46667-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.372086 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.372068    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.372320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.372299    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.372418 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.372339    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba4c3f49-4422-483f-b24f-41545ed46667-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.372418 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.372380    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:41.372502 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.372452    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.372549 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.372457    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:41.372584 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.372553    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:41.415912 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.415878    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"]
Oct 30 15:50:41.421219 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.421193    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.443724 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.443695    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"]
Oct 30 15:50:41.444012 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.443992    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-b24e7b-openshift-default-dockercfg-fsj9p\""
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.473499    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.473552    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba4c3f49-4422-483f-b24f-41545ed46667-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.473614    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.473679    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.473708    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bmlx\" (UniqueName: \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-kube-api-access-8bmlx\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.473754    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.473805    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.473857    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba4c3f49-4422-483f-b24f-41545ed46667-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.473916    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.474377    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.474660    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.475134    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.475583    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.476434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.475998    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba4c3f49-4422-483f-b24f-41545ed46667-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.477446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.477423    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.480410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.479878    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba4c3f49-4422-483f-b24f-41545ed46667-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.500613 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.500572    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bmlx\" (UniqueName: \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-kube-api-access-8bmlx\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.507982 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.507944    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.575361 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.575301    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.575830 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.575455    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b400747-55f6-49cb-a8dc-3b547323787e-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.575830 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.575600    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.575830 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.575635    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6ftm\" (UniqueName: \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-kube-api-access-m6ftm\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.575830 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.575667    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.575830 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.575809    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.576140 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.575848    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.576140 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.575897    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b400747-55f6-49cb-a8dc-3b547323787e-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.576140 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.575958    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.586366 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.586317    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:41.621561 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.620768    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" event={"ID":"876862ba-27c9-4c8b-b440-93ec2241b948","Type":"ContainerStarted","Data":"8a8dc2bc4742a8d962906026407881e10956fca1de9c7aeb341879a4dca5f88c"}
Oct 30 15:50:41.621561 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.620827    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" event={"ID":"876862ba-27c9-4c8b-b440-93ec2241b948","Type":"ContainerStarted","Data":"878e9eeb8399238701fd3e8e62f708c1d13edfe91579bf0bbd37cb8cda2efef0"}
Oct 30 15:50:41.624567 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.624531    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" event={"ID":"35c54ae6-a861-4efc-8398-78b4610e0c40","Type":"ContainerStarted","Data":"4b3eb7968795a66609551c011975cb8db7c07e132f339f1edb37636992114d30"}
Oct 30 15:50:41.624724 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.624599    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" event={"ID":"35c54ae6-a861-4efc-8398-78b4610e0c40","Type":"ContainerStarted","Data":"bb4f9519ce66b5d91252db5815c1765ee988ab7e7dce8b32707806374f880317"}
Oct 30 15:50:41.677222 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.677183    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.677461 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.677440    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.677680 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.677664    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b400747-55f6-49cb-a8dc-3b547323787e-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.677826 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.677812    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.677965 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.677950    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.678197 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.678179    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b400747-55f6-49cb-a8dc-3b547323787e-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.678419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.678402    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.678541 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.678524    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6ftm\" (UniqueName: \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-kube-api-access-m6ftm\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.678875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.678856    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.679610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.679544    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" podStartSLOduration=1.679527719 podStartE2EDuration="1.679527719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:41.678850771 +0000 UTC m=+751.114455119" watchObservedRunningTime="2026-10-30 15:50:41.679527719 +0000 UTC m=+751.115132068"
Oct 30 15:50:41.679719 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.679614    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.679719 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.679687    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.683386 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.680294    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.683386 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.680589    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.683386 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.680766    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b400747-55f6-49cb-a8dc-3b547323787e-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.688566 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.688442    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b400747-55f6-49cb-a8dc-3b547323787e-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.693746 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.693637    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.702014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.701975    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.703180 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.703152    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6ftm\" (UniqueName: \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-kube-api-access-m6ftm\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.738850 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.738740    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podStartSLOduration=1.73872345 podStartE2EDuration="1.73872345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:41.735433379 +0000 UTC m=+751.171037733" watchObservedRunningTime="2026-10-30 15:50:41.73872345 +0000 UTC m=+751.174327798"
Oct 30 15:50:41.739068 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.739052    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:41.880888 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.880837    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:41.908419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.908393    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"]
Oct 30 15:50:41.909805 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:50:41.909769    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4c3f49_4422_483f_b24f_41545ed46667.slice/crio-6c650c45f2bcec31d5c31a0f0667a8078514d7f3fc839e210fa6830dd2b55b47 WatchSource:0}: Error finding container 6c650c45f2bcec31d5c31a0f0667a8078514d7f3fc839e210fa6830dd2b55b47: Status 404 returned error can't find the container with id 6c650c45f2bcec31d5c31a0f0667a8078514d7f3fc839e210fa6830dd2b55b47
Oct 30 15:50:41.912655 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.912613    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:41.912738 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.912709    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:41.912785 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:41.912764    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:42.049206 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.049181    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:42.049758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.049686    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"]
Oct 30 15:50:42.055515 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.055475    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:42.055626 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.055565    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:42.055626 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.055609    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:50:42.057794 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.057770    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:42.630703 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.630665    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" event={"ID":"ba4c3f49-4422-483f-b24f-41545ed46667","Type":"ContainerStarted","Data":"971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143"}
Oct 30 15:50:42.631156 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.630727    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" event={"ID":"ba4c3f49-4422-483f-b24f-41545ed46667","Type":"ContainerStarted","Data":"6c650c45f2bcec31d5c31a0f0667a8078514d7f3fc839e210fa6830dd2b55b47"}
Oct 30 15:50:42.632405 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.632370    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" event={"ID":"8b400747-55f6-49cb-a8dc-3b547323787e","Type":"ContainerStarted","Data":"732a4ec34374b4b62e5d300dba5c04827d45f2656296acf47a34d626f789c7fb"}
Oct 30 15:50:42.632548 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.632424    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" event={"ID":"8b400747-55f6-49cb-a8dc-3b547323787e","Type":"ContainerStarted","Data":"dbf0f258d0d40df38df246977b0ad459e5411775dc100c184346e071ef79c329"}
Oct 30 15:50:42.632762 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.632738    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:42.633683 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.633664    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:50:42.677064 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.677015    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" podStartSLOduration=1.6770011070000002 podStartE2EDuration="1.677001107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:42.675264481 +0000 UTC m=+752.110868828" watchObservedRunningTime="2026-10-30 15:50:42.677001107 +0000 UTC m=+752.112605455"
Oct 30 15:50:42.721236 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.721179    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podStartSLOduration=1.721160266 podStartE2EDuration="1.721160266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:42.720658253 +0000 UTC m=+752.156262601" watchObservedRunningTime="2026-10-30 15:50:42.721160266 +0000 UTC m=+752.156764617"
Oct 30 15:50:42.740127 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.740099    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:42.902170 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:42.902136    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:43.587128 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:43.587092    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:43.592231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:43.592204    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:43.637569 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:43.637538    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:43.638046 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:43.637616    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:43.638202 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:43.638177    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:50:43.638339 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:43.638304    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:50:43.746601 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:43.746568    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:44.646641 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:44.646613    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:44.647540 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:44.647509    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:50:47.964055 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:50:47.964021    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:50:50.374985 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.374954    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"]
Oct 30 15:50:50.386054 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.386025    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:50:50.397389 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.397330    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"]
Oct 30 15:50:50.407683 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.407655    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-9e2b29\""
Oct 30 15:50:50.483522 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.483490    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74988\" (UniqueName: \"kubernetes.io/projected/547f6ffe-6f5a-4dac-8521-4e92df79f82e-kube-api-access-74988\") pod \"payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4\" (UID: \"547f6ffe-6f5a-4dac-8521-4e92df79f82e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:50:50.483704 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.483535    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547f6ffe-6f5a-4dac-8521-4e92df79f82e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4\" (UID: \"547f6ffe-6f5a-4dac-8521-4e92df79f82e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:50:50.585264 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.585217    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74988\" (UniqueName: \"kubernetes.io/projected/547f6ffe-6f5a-4dac-8521-4e92df79f82e-kube-api-access-74988\") pod \"payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4\" (UID: \"547f6ffe-6f5a-4dac-8521-4e92df79f82e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:50:50.585466 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.585280    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547f6ffe-6f5a-4dac-8521-4e92df79f82e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4\" (UID: \"547f6ffe-6f5a-4dac-8521-4e92df79f82e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:50:50.586255 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.586230    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547f6ffe-6f5a-4dac-8521-4e92df79f82e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4\" (UID: \"547f6ffe-6f5a-4dac-8521-4e92df79f82e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:50:50.609505 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.609466    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74988\" (UniqueName: \"kubernetes.io/projected/547f6ffe-6f5a-4dac-8521-4e92df79f82e-kube-api-access-74988\") pod \"payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4\" (UID: \"547f6ffe-6f5a-4dac-8521-4e92df79f82e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:50:50.702758 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:50.702725    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:50:51.000254 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.000228    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"]
Oct 30 15:50:51.001834 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:50:51.001800    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547f6ffe_6f5a_4dac_8521_4e92df79f82e.slice/crio-48fa8743fea9252503460527b21b4082f4c8614f14c87e50a0351e7df3fc9765 WatchSource:0}: Error finding container 48fa8743fea9252503460527b21b4082f4c8614f14c87e50a0351e7df3fc9765: Status 404 returned error can't find the container with id 48fa8743fea9252503460527b21b4082f4c8614f14c87e50a0351e7df3fc9765
Oct 30 15:50:51.194624 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.194588    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"]
Oct 30 15:50:51.199318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.199297    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:50:51.220995 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.220481    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"]
Oct 30 15:50:51.228501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.228478    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-a2b3f1\""
Oct 30 15:50:51.291881 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.291762    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvhv9\" (UniqueName: \"kubernetes.io/projected/2598a7ca-70ff-4bd2-ae89-fa742412fa98-kube-api-access-xvhv9\") pod \"payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg\" (UID: \"2598a7ca-70ff-4bd2-ae89-fa742412fa98\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:50:51.291881 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.291809    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2598a7ca-70ff-4bd2-ae89-fa742412fa98-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg\" (UID: \"2598a7ca-70ff-4bd2-ae89-fa742412fa98\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:50:51.393384 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.393326    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvhv9\" (UniqueName: \"kubernetes.io/projected/2598a7ca-70ff-4bd2-ae89-fa742412fa98-kube-api-access-xvhv9\") pod \"payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg\" (UID: \"2598a7ca-70ff-4bd2-ae89-fa742412fa98\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:50:51.393384 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.393380    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2598a7ca-70ff-4bd2-ae89-fa742412fa98-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg\" (UID: \"2598a7ca-70ff-4bd2-ae89-fa742412fa98\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:50:51.394160 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.394133    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2598a7ca-70ff-4bd2-ae89-fa742412fa98-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg\" (UID: \"2598a7ca-70ff-4bd2-ae89-fa742412fa98\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:50:51.418959 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.418927    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvhv9\" (UniqueName: \"kubernetes.io/projected/2598a7ca-70ff-4bd2-ae89-fa742412fa98-kube-api-access-xvhv9\") pod \"payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg\" (UID: \"2598a7ca-70ff-4bd2-ae89-fa742412fa98\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:50:51.512956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.512695    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:50:51.677456 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.677416    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4" event={"ID":"547f6ffe-6f5a-4dac-8521-4e92df79f82e","Type":"ContainerStarted","Data":"48fa8743fea9252503460527b21b4082f4c8614f14c87e50a0351e7df3fc9765"}
Oct 30 15:50:51.824893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:51.824830    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"]
Oct 30 15:50:51.825939 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:50:51.825903    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2598a7ca_70ff_4bd2_ae89_fa742412fa98.slice/crio-bafc351362365e3432ca256a886298a23bbcb87eaac943f24a6696691d0d634d WatchSource:0}: Error finding container bafc351362365e3432ca256a886298a23bbcb87eaac943f24a6696691d0d634d: Status 404 returned error can't find the container with id bafc351362365e3432ca256a886298a23bbcb87eaac943f24a6696691d0d634d
Oct 30 15:50:52.683180 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:52.683146    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg" event={"ID":"2598a7ca-70ff-4bd2-ae89-fa742412fa98","Type":"ContainerStarted","Data":"bafc351362365e3432ca256a886298a23bbcb87eaac943f24a6696691d0d634d"}
Oct 30 15:50:53.272845 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.272801    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"]
Oct 30 15:50:53.282513 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.282483    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:50:53.296982 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.296951    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"]
Oct 30 15:50:53.307736 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.307708    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-b24e7b\""
Oct 30 15:50:53.307736 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.307713    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-b24e7b-dockercfg-54ktn\""
Oct 30 15:50:53.337458 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.337421    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f4nb\" (UniqueName: \"kubernetes.io/projected/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-kube-api-access-2f4nb\") pod \"payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf\" (UID: \"133ba3b4-6cf3-42f1-a8ed-fa8285df655b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:50:53.337626 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.337588    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf\" (UID: \"133ba3b4-6cf3-42f1-a8ed-fa8285df655b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:50:53.438800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.438767    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf\" (UID: \"133ba3b4-6cf3-42f1-a8ed-fa8285df655b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:50:53.438967 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.438847    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f4nb\" (UniqueName: \"kubernetes.io/projected/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-kube-api-access-2f4nb\") pod \"payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf\" (UID: \"133ba3b4-6cf3-42f1-a8ed-fa8285df655b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:50:53.439647 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.439624    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf\" (UID: \"133ba3b4-6cf3-42f1-a8ed-fa8285df655b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:50:53.505992 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.502065    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f4nb\" (UniqueName: \"kubernetes.io/projected/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-kube-api-access-2f4nb\") pod \"payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf\" (UID: \"133ba3b4-6cf3-42f1-a8ed-fa8285df655b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:50:53.605756 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.604851    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:50:53.933416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:53.933383    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"]
Oct 30 15:50:53.934772 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:50:53.934744    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133ba3b4_6cf3_42f1_a8ed_fa8285df655b.slice/crio-57a6b4c47e078e61b3c0f613e0d1790e4db479748d1d2f070680ec1f6b9ff905 WatchSource:0}: Error finding container 57a6b4c47e078e61b3c0f613e0d1790e4db479748d1d2f070680ec1f6b9ff905: Status 404 returned error can't find the container with id 57a6b4c47e078e61b3c0f613e0d1790e4db479748d1d2f070680ec1f6b9ff905
Oct 30 15:50:54.262306 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.262209    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"]
Oct 30 15:50:54.272325 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.272299    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:50:54.288105 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.288073    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"]
Oct 30 15:50:54.327383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.313796    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-c129d1-dockercfg-f2spn\""
Oct 30 15:50:54.327383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.317298    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-c129d1\""
Oct 30 15:50:54.349989 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.349954    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgdnx\" (UniqueName: \"kubernetes.io/projected/d18ea2be-fa5a-45c8-8589-e081450576d4-kube-api-access-zgdnx\") pod \"payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb\" (UID: \"d18ea2be-fa5a-45c8-8589-e081450576d4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:50:54.350076 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.350058    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d18ea2be-fa5a-45c8-8589-e081450576d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb\" (UID: \"d18ea2be-fa5a-45c8-8589-e081450576d4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:50:54.451332 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.451297    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d18ea2be-fa5a-45c8-8589-e081450576d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb\" (UID: \"d18ea2be-fa5a-45c8-8589-e081450576d4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:50:54.451544 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.451466    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgdnx\" (UniqueName: \"kubernetes.io/projected/d18ea2be-fa5a-45c8-8589-e081450576d4-kube-api-access-zgdnx\") pod \"payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb\" (UID: \"d18ea2be-fa5a-45c8-8589-e081450576d4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:50:54.452035 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.452015    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d18ea2be-fa5a-45c8-8589-e081450576d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb\" (UID: \"d18ea2be-fa5a-45c8-8589-e081450576d4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:50:54.474949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.474909    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgdnx\" (UniqueName: \"kubernetes.io/projected/d18ea2be-fa5a-45c8-8589-e081450576d4-kube-api-access-zgdnx\") pod \"payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb\" (UID: \"d18ea2be-fa5a-45c8-8589-e081450576d4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:50:54.588399 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.588279    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:50:54.705153 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.705118    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg" event={"ID":"2598a7ca-70ff-4bd2-ae89-fa742412fa98","Type":"ContainerStarted","Data":"239bb281af2a6ddefa6abe67a56aa4ed1c5d508068bee9d2a3886ff13ad0029e"}
Oct 30 15:50:54.705376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.705261    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:50:54.709887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.709852    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf" event={"ID":"133ba3b4-6cf3-42f1-a8ed-fa8285df655b","Type":"ContainerStarted","Data":"772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0"}
Oct 30 15:50:54.710013 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.709911    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf" event={"ID":"133ba3b4-6cf3-42f1-a8ed-fa8285df655b","Type":"ContainerStarted","Data":"57a6b4c47e078e61b3c0f613e0d1790e4db479748d1d2f070680ec1f6b9ff905"}
Oct 30 15:50:54.710013 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.709937    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:50:54.711566 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.711542    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4" event={"ID":"547f6ffe-6f5a-4dac-8521-4e92df79f82e","Type":"ContainerStarted","Data":"92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f"}
Oct 30 15:50:54.711690 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.711597    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:50:54.746191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.746144    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg" podStartSLOduration=1.249277573 podStartE2EDuration="3.7461284s" totalImagesPullingTime="2.496850827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:54.745039109 +0000 UTC m=+764.180643458" watchObservedRunningTime="2026-10-30 15:50:54.7461284 +0000 UTC m=+764.181732747"
Oct 30 15:50:54.786162 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.786112    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf" podStartSLOduration=1.7860970250000001 podStartE2EDuration="1.786097025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:54.785273411 +0000 UTC m=+764.220877760" watchObservedRunningTime="2026-10-30 15:50:54.786097025 +0000 UTC m=+764.221701422"
Oct 30 15:50:54.825731 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.825681    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4" podStartSLOduration=2.113572627 podStartE2EDuration="4.825665957s" totalImagesPullingTime="2.71209333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:54.824270873 +0000 UTC m=+764.259875225" watchObservedRunningTime="2026-10-30 15:50:54.825665957 +0000 UTC m=+764.261270305"
Oct 30 15:50:54.890798 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:54.890771    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"]
Oct 30 15:50:54.892107 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:50:54.892083    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18ea2be_fa5a_45c8_8589_e081450576d4.slice/crio-a76e69b9a545c387f27ff86b366d52e15805c41b307c5cb3ceb725e01446e2a4 WatchSource:0}: Error finding container a76e69b9a545c387f27ff86b366d52e15805c41b307c5cb3ceb725e01446e2a4: Status 404 returned error can't find the container with id a76e69b9a545c387f27ff86b366d52e15805c41b307c5cb3ceb725e01446e2a4
Oct 30 15:50:55.373840 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.373810    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"]
Oct 30 15:50:55.377433 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.377413    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.395939 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.395913    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-b63c3c-serving-cert\""
Oct 30 15:50:55.406507 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.405014    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 15:50:55.406507 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.405043    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 15:50:55.406507 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.405126    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 15:50:55.407633 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.407604    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"]
Oct 30 15:50:55.408030 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.407986    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gp5lt\""
Oct 30 15:50:55.460295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.460262    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-maas-api-tls\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.460507 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.460390    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9d8649f-413e-486f-9499-aac47e0d7723-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.460573 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.460552    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdxtr\" (UniqueName: \"kubernetes.io/projected/c9d8649f-413e-486f-9499-aac47e0d7723-kube-api-access-cdxtr\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.460610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.460594    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-metrics-certs\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.562028 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.561980    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-maas-api-tls\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.562201 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.562063    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9d8649f-413e-486f-9499-aac47e0d7723-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.562201 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.562180    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdxtr\" (UniqueName: \"kubernetes.io/projected/c9d8649f-413e-486f-9499-aac47e0d7723-kube-api-access-cdxtr\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.562273 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.562222    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-metrics-certs\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.562877 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.562826    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9d8649f-413e-486f-9499-aac47e0d7723-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.564563 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.564542    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-maas-api-tls\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.564667 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.564646    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-metrics-certs\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.586062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.586023    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdxtr\" (UniqueName: \"kubernetes.io/projected/c9d8649f-413e-486f-9499-aac47e0d7723-kube-api-access-cdxtr\") pod \"maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.690498 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.690442    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:55.718725 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.718683    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb" event={"ID":"d18ea2be-fa5a-45c8-8589-e081450576d4","Type":"ContainerStarted","Data":"945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0"}
Oct 30 15:50:55.718893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.718743    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb" event={"ID":"d18ea2be-fa5a-45c8-8589-e081450576d4","Type":"ContainerStarted","Data":"a76e69b9a545c387f27ff86b366d52e15805c41b307c5cb3ceb725e01446e2a4"}
Oct 30 15:50:55.719083 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.719063    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:50:55.994705 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.994653    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb" podStartSLOduration=1.9946319049999999 podStartE2EDuration="1.994631905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:55.777596937 +0000 UTC m=+765.213201286" watchObservedRunningTime="2026-10-30 15:50:55.994631905 +0000 UTC m=+765.430236254"
Oct 30 15:50:55.995195 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:55.995176    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"]
Oct 30 15:50:55.995907 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:50:55.995889    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d8649f_413e_486f_9499_aac47e0d7723.slice/crio-02c37647bf06bd2ba7e9f78d676255ed7d6f280ce58154a2316c19f5a6f4c30e WatchSource:0}: Error finding container 02c37647bf06bd2ba7e9f78d676255ed7d6f280ce58154a2316c19f5a6f4c30e: Status 404 returned error can't find the container with id 02c37647bf06bd2ba7e9f78d676255ed7d6f280ce58154a2316c19f5a6f4c30e
Oct 30 15:50:56.726722 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:56.726670    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb" event={"ID":"c9d8649f-413e-486f-9499-aac47e0d7723","Type":"ContainerStarted","Data":"02c37647bf06bd2ba7e9f78d676255ed7d6f280ce58154a2316c19f5a6f4c30e"}
Oct 30 15:50:59.770765 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:59.770730    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb" event={"ID":"c9d8649f-413e-486f-9499-aac47e0d7723","Type":"ContainerStarted","Data":"d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58"}
Oct 30 15:50:59.771191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:59.770874    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:50:59.812722 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:50:59.812663    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb" podStartSLOduration=1.808379499 podStartE2EDuration="4.81265022s" totalImagesPullingTime="3.004270721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:50:59.812621038 +0000 UTC m=+769.248225387" watchObservedRunningTime="2026-10-30 15:50:59.81265022 +0000 UTC m=+769.248254569"
Oct 30 15:51:02.914836 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:02.914798    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"]
Oct 30 15:51:02.918713 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:02.918694    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:02.966412 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:02.965723    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"]
Oct 30 15:51:02.988400 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:02.986837    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-c129d1-serving-cert\""
Oct 30 15:51:03.015085 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.014627    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f1199e1-3446-4e9c-acca-96e7572c1952-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.015262 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.015218    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-maas-api-tls\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.015376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.015298    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scz4g\" (UniqueName: \"kubernetes.io/projected/0f1199e1-3446-4e9c-acca-96e7572c1952-kube-api-access-scz4g\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.015498 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.015474    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-metrics-certs\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.116609 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.116566    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f1199e1-3446-4e9c-acca-96e7572c1952-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.116609 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.116613    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-maas-api-tls\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.116873 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.116783    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scz4g\" (UniqueName: \"kubernetes.io/projected/0f1199e1-3446-4e9c-acca-96e7572c1952-kube-api-access-scz4g\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.116873 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.116848    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-metrics-certs\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.117270 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.117245    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f1199e1-3446-4e9c-acca-96e7572c1952-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.119398 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.119330    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-maas-api-tls\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.119973 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.119950    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-metrics-certs\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.163245 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.163167    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scz4g\" (UniqueName: \"kubernetes.io/projected/0f1199e1-3446-4e9c-acca-96e7572c1952-kube-api-access-scz4g\") pod \"maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.251426 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.239926    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:03.587655 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.587623    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"]
Oct 30 15:51:03.588008 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:51:03.587987    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1199e1_3446_4e9c_acca_96e7572c1952.slice/crio-a382a49cbf5890648314dc09a53a03e6bd72cda99203a8d8f6bb0e6903f17d09 WatchSource:0}: Error finding container a382a49cbf5890648314dc09a53a03e6bd72cda99203a8d8f6bb0e6903f17d09: Status 404 returned error can't find the container with id a382a49cbf5890648314dc09a53a03e6bd72cda99203a8d8f6bb0e6903f17d09
Oct 30 15:51:03.798163 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:03.798120    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv" event={"ID":"0f1199e1-3446-4e9c-acca-96e7572c1952","Type":"ContainerStarted","Data":"a382a49cbf5890648314dc09a53a03e6bd72cda99203a8d8f6bb0e6903f17d09"}
Oct 30 15:51:04.805343 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:04.805305    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv" event={"ID":"0f1199e1-3446-4e9c-acca-96e7572c1952","Type":"ContainerStarted","Data":"2dc2de6b4590b2864599571e88c099376005c2fdb43b659f12c243ff8a03db10"}
Oct 30 15:51:04.852370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:04.851376    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv" podStartSLOduration=2.441470966 podStartE2EDuration="2.851336992s" totalImagesPullingTime="409.866026ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:04.850773927 +0000 UTC m=+774.286378275" watchObservedRunningTime="2026-10-30 15:51:04.851336992 +0000 UTC m=+774.286941343"
Oct 30 15:51:05.723137 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:05.722452    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:51:05.723687 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:05.723662    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:51:05.724316 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:05.723883    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:51:05.791722 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:05.791689    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:51:05.811155 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:05.811122    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:06.755581 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:06.755546    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:51:06.967612 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:06.966206    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"]
Oct 30 15:51:06.972316 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:06.972281    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:06.977155 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:06.975904    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"]
Oct 30 15:51:06.993803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:06.991447    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-040f9a\"/\"openshift-service-ca.crt\""
Oct 30 15:51:06.993803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:06.991608    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-040f9a\"/\"default-dockercfg-7zh4d\""
Oct 30 15:51:06.993803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:06.991719    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-040f9a\"/\"simulator-w3-040f9a-kserve-self-signed-certs\""
Oct 30 15:51:06.993803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:06.991803    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-040f9a\"/\"kube-root-ca.crt\""
Oct 30 15:51:07.006179 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.006094    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-home\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.006531 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.006481    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lcp9\" (UniqueName: \"kubernetes.io/projected/1ae2931d-aa72-4b77-bd29-78d29060ff8f-kube-api-access-2lcp9\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.006531 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.006525    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-dshm\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.006531 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.006602    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-model-cache\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.006531 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.006632    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tmp-dir\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.006531 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.006655    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tls-certs\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.107887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.107835    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-home\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.108062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.107917    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lcp9\" (UniqueName: \"kubernetes.io/projected/1ae2931d-aa72-4b77-bd29-78d29060ff8f-kube-api-access-2lcp9\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.108062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.107936    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-dshm\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.108062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.107981    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-model-cache\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.108062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.108002    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tmp-dir\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.108062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.108018    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tls-certs\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.108381 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.108336    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-home\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.108514 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.108492    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tmp-dir\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.108634 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.108591    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-model-cache\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.110531 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.110500    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-dshm\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.110625 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.110594    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tls-certs\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.156784 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.156735    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lcp9\" (UniqueName: \"kubernetes.io/projected/1ae2931d-aa72-4b77-bd29-78d29060ff8f-kube-api-access-2lcp9\") pod \"simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") " pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.294932 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.294833    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:07.636825 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.634721    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"]
Oct 30 15:51:07.826431 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.826041    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" event={"ID":"1ae2931d-aa72-4b77-bd29-78d29060ff8f","Type":"ContainerStarted","Data":"d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608"}
Oct 30 15:51:07.826431 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.826098    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" event={"ID":"1ae2931d-aa72-4b77-bd29-78d29060ff8f","Type":"ContainerStarted","Data":"f382de9295ec9995fd761507b6c023d87ddaeafc337ee0e05bb067e09f319f5e"}
Oct 30 15:51:07.828294 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.828267    4585 generic.go:356] "Generic (PLEG): container finished" podID="876862ba-27c9-4c8b-b440-93ec2241b948" containerID="8a8dc2bc4742a8d962906026407881e10956fca1de9c7aeb341879a4dca5f88c" exitCode=0
Oct 30 15:51:07.828436 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.828293    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" event={"ID":"876862ba-27c9-4c8b-b440-93ec2241b948","Type":"ContainerDied","Data":"8a8dc2bc4742a8d962906026407881e10956fca1de9c7aeb341879a4dca5f88c"}
Oct 30 15:51:07.838430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.828736    4585 scope.go:122] "RemoveContainer" containerID="8a8dc2bc4742a8d962906026407881e10956fca1de9c7aeb341879a4dca5f88c"
Oct 30 15:51:07.853853 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.845660    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:07.853853 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.845755    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:07.853853 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.845800    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:07.887450 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:07.885370    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podStartSLOduration=1.885340116 podStartE2EDuration="1.885340116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:07.881991838 +0000 UTC m=+777.317596187" watchObservedRunningTime="2026-10-30 15:51:07.885340116 +0000 UTC m=+777.320944468"
Oct 30 15:51:08.295288 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:08.295227    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:08.876375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:08.873452    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" event={"ID":"876862ba-27c9-4c8b-b440-93ec2241b948","Type":"ContainerStarted","Data":"cf199c732f2ba03c1cbc0781f88a36353c7972bb86030c70603749dcabaf97dd"}
Oct 30 15:51:09.051108 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:09.050665    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:51:09.051108 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:09.050797    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 15:51:09.051108 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:09.050841    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 15:51:10.052934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.052897    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 15:51:10.053572 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.053531    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 15:51:10.803401 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.797975    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"]
Oct 30 15:51:10.809934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.809895    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"]
Oct 30 15:51:10.810309 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.810288    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:10.823313 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:51:10.823265    4585 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547f6ffe_6f5a_4dac_8521_4e92df79f82e.slice/crio-48fa8743fea9252503460527b21b4082f4c8614f14c87e50a0351e7df3fc9765\": RecentStats: unable to find data in memory cache]"
Oct 30 15:51:10.913546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.913511    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:10.913709 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.913562    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v59pd\" (UniqueName: \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-kube-api-access-v59pd\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:10.913709 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.913678    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:10.913799 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.913710    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:10.913799 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.913726    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:10.913799 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.913752    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:10.913917 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.913801    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:10.913917 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.913896    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2c5fdab-3219-4561-a188-9ec859fb77fc-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:10.913994 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:10.913925    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.014970 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.014935    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.015168 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.014989    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v59pd\" (UniqueName: \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-kube-api-access-v59pd\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.015168 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.015045    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.015168 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.015072    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.015168 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.015094    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.015168 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.015128    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.015168 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.015148    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.015507 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.015184    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2c5fdab-3219-4561-a188-9ec859fb77fc-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.015507 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.015225    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.015507 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.015408    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.016093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.016063    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.016576 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.016409    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.016786 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.016741    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.017072 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.017046    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2c5fdab-3219-4561-a188-9ec859fb77fc-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.018252 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.018203    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.018626 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.018600    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.046373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.042290    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.046373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.045478    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v59pd\" (UniqueName: \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-kube-api-access-v59pd\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.050407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.049427    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 15:51:11.050407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.049481    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 15:51:11.069259 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.066068    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"]
Oct 30 15:51:11.075370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.071711    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.108319 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.108229    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"]
Oct 30 15:51:11.125480 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.125442    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.125669 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.125505    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.125669 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.125533    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2zp6\" (UniqueName: \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-kube-api-access-l2zp6\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.125669 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.125575    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.125669 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.125606    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.125669 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.125659    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.126015 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.125701    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.126015 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.125812    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.126015 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.125849    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.144090 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.144030    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-a2b3f1-openshift-default-dockercfg-jmzld\""
Oct 30 15:51:11.182164 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.181763    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:11.227159 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.227121    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.227288 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.227185    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.227288 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.227229    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.227288 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.227271    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.227497 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.227300    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2zp6\" (UniqueName: \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-kube-api-access-l2zp6\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.227497 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.227365    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.227497 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.227403    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.227497 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.227468    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.227707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.227514    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.242952 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.228527    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.242952 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.228781    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.242952 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.228996    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.242952 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.229229    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.242952 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.238024    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.280373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.271505    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.280373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.280073    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2zp6\" (UniqueName: \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-kube-api-access-l2zp6\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.303223 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.302991    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.326418 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.325233    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.514634 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.514594    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:11.821232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.821199    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"]
Oct 30 15:51:11.823911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.822652    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:11.823911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.822740    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:11.823911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.822783    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:11.874431 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.873562    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:51:11.912089 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:11.912057    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" event={"ID":"f2c5fdab-3219-4561-a188-9ec859fb77fc","Type":"ContainerStarted","Data":"36102f713a5a82cfcd36e4b28aa69652eabb98dd97cda158b0ebf12d86d3714f"}
Oct 30 15:51:12.051408 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:12.050970    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 15:51:12.051408 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:12.051026    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 15:51:12.092665 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:12.085990    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"]
Oct 30 15:51:12.105627 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:12.105307    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:12.114773 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:12.114524    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:12.114773 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:12.114604    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:12.636549 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:12.636452    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:51:12.957370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:12.953170    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" event={"ID":"f2c5fdab-3219-4561-a188-9ec859fb77fc","Type":"ContainerStarted","Data":"79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360"}
Oct 30 15:51:12.957370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:12.956287    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" event={"ID":"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb","Type":"ContainerStarted","Data":"65269abff28a2f3289a78852ed93e851e135149e0cd298d24093560c9c72b07f"}
Oct 30 15:51:12.957370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:12.956342    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" event={"ID":"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb","Type":"ContainerStarted","Data":"db0af2f30a38ff24eecaf87f02c9ae164bcb44a783881b961e956b0f6b46f068"}
Oct 30 15:51:13.034466 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:13.026540    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" podStartSLOduration=3.026519883 podStartE2EDuration="3.026519883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:13.025412804 +0000 UTC m=+782.461017153" watchObservedRunningTime="2026-10-30 15:51:13.026519883 +0000 UTC m=+782.462124233"
Oct 30 15:51:13.057237 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:13.057202    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 15:51:13.058570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:13.058534    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 15:51:13.094454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:13.094396    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" podStartSLOduration=2.094374816 podStartE2EDuration="2.094374816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:13.092935636 +0000 UTC m=+782.528539984" watchObservedRunningTime="2026-10-30 15:51:13.094374816 +0000 UTC m=+782.529979165"
Oct 30 15:51:13.191632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:13.191597    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:13.191947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:13.191926    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 15:51:13.192112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:13.192087    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 15:51:13.523995 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:13.523538    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:13.523995 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:13.523681    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 15:51:13.523995 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:13.523725    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 15:51:14.053286 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:14.052830    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 15:51:14.053286 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:14.052911    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 15:51:14.185846 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:14.185806    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 15:51:14.186972 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:14.186495    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 15:51:14.517389 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:14.515488    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 15:51:14.517389 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:14.515549    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 15:51:14.706958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:14.706426    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:51:14.706958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:14.706488    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:51:15.188386 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:15.183780    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 15:51:15.188386 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:15.183841    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 15:51:15.411133 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:15.409648    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:51:15.416650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:15.416579    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 15:51:15.563458 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:15.517382    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 15:51:15.563458 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:15.517441    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 15:51:15.910441 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:15.906471    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:51:15.910441 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:15.906536    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:51:16.187720 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:16.187630    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 15:51:16.187971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:16.187943    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 15:51:16.515833 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:16.515759    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 15:51:16.516280 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:16.516242    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 15:51:17.197319 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:17.196823    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 15:51:17.197319 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:17.196890    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 15:51:17.298259 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:17.297422    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:17.520430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:17.516011    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 15:51:17.520430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:17.516072    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 15:51:17.916373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:17.916166    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:18.221434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:18.193763    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 15:51:18.221434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:18.193825    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 15:51:18.221434 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:51:18.201142    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:51:18.447398 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:18.439727    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:51:18.547455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:18.531436    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 15:51:18.547455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:18.531503    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 15:51:19.182547 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:19.182509    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 15:51:19.182956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:19.182925    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 15:51:19.516600 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:19.516069    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 15:51:19.516600 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:19.516131    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 15:51:20.189307 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:20.189266    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:20.198370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:20.190671    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:20.198370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:20.191982    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:51:20.523295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:20.522022    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:20.523295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:20.523184    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:20.524385 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:20.524316    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:51:23.141399 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:23.141326    4585 generic.go:356] "Generic (PLEG): container finished" podID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerID="4b3eb7968795a66609551c011975cb8db7c07e132f339f1edb37636992114d30" exitCode=0
Oct 30 15:51:23.141844 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:23.141397    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" event={"ID":"35c54ae6-a861-4efc-8398-78b4610e0c40","Type":"ContainerDied","Data":"4b3eb7968795a66609551c011975cb8db7c07e132f339f1edb37636992114d30"}
Oct 30 15:51:23.141844 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:23.141783    4585 scope.go:122] "RemoveContainer" containerID="4b3eb7968795a66609551c011975cb8db7c07e132f339f1edb37636992114d30"
Oct 30 15:51:23.143171 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:23.143137    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:23.143238 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:23.143209    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:23.143273 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:23.143245    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:24.165600 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.165562    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" event={"ID":"35c54ae6-a861-4efc-8398-78b4610e0c40","Type":"ContainerStarted","Data":"01c1f11005d4e9e6afbffc4b6011cf187e8034ed9d3a0abf152462d5529a9559"}
Oct 30 15:51:24.333668 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.333630    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"]
Oct 30 15:51:24.363190 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.363157    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.364765 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.364536    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"]
Oct 30 15:51:24.378870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.378848    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b63c3c\"/\"kube-root-ca.crt\""
Oct 30 15:51:24.379463 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.379428    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b63c3c\"/\"openshift-service-ca.crt\""
Oct 30 15:51:24.379535 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.379523    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b63c3c\"/\"simulator-w0-b63c3c-kserve-self-signed-certs\""
Oct 30 15:51:24.379591 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.379581    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b63c3c\"/\"default-dockercfg-wksm9\""
Oct 30 15:51:24.421709 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.421623    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-home\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.421709 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.421680    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tls-certs\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.421954 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.421772    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-model-cache\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.421954 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.421828    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tmp-dir\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.421954 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.421858    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-dshm\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.422112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.421958    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdsrh\" (UniqueName: \"kubernetes.io/projected/c0ae31d1-0e25-441b-8257-4871a18ed6ee-kube-api-access-zdsrh\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.523056 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.523011    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-home\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.523267 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.523104    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tls-certs\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.523267 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.523143    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-model-cache\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.523267 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.523180    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tmp-dir\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.523267 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.523218    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-dshm\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.523520 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.523293    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdsrh\" (UniqueName: \"kubernetes.io/projected/c0ae31d1-0e25-441b-8257-4871a18ed6ee-kube-api-access-zdsrh\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.523520 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.523501    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-home\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.523785 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.523752    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tmp-dir\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.523926 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.523906    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-model-cache\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.525703 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.525681    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-dshm\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.526108 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.526089    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tls-certs\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.560372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.560260    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdsrh\" (UniqueName: \"kubernetes.io/projected/c0ae31d1-0e25-441b-8257-4871a18ed6ee-kube-api-access-zdsrh\") pod \"simulator-w0-b63c3c-kserve-5998f5d59-562jm\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") " pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.697845 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.697777    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:24.880462 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.880392    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:51:24.881503 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.881476    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 15:51:24.881632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:24.881541    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 15:51:25.022374 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:51:25.022326    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0ae31d1_0e25_441b_8257_4871a18ed6ee.slice/crio-046f75a42df38c3e7006e037e67873e32671f79010604a38efc3cc8f7858d620 WatchSource:0}: Error finding container 046f75a42df38c3e7006e037e67873e32671f79010604a38efc3cc8f7858d620: Status 404 returned error can't find the container with id 046f75a42df38c3e7006e037e67873e32671f79010604a38efc3cc8f7858d620
Oct 30 15:51:25.022487 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.022395    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"]
Oct 30 15:51:25.173934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.173897    4585 generic.go:356] "Generic (PLEG): container finished" podID="ba4c3f49-4422-483f-b24f-41545ed46667" containerID="971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143" exitCode=0
Oct 30 15:51:25.174445 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.173972    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" event={"ID":"ba4c3f49-4422-483f-b24f-41545ed46667","Type":"ContainerDied","Data":"971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143"}
Oct 30 15:51:25.174520 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.174505    4585 scope.go:122] "RemoveContainer" containerID="971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143"
Oct 30 15:51:25.176018 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.175952    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" event={"ID":"c0ae31d1-0e25-441b-8257-4871a18ed6ee","Type":"ContainerStarted","Data":"cbfd39155c83bfe56b02f929853592cc7947679e828b63c4290177665148b272"}
Oct 30 15:51:25.176018 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.175992    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" event={"ID":"c0ae31d1-0e25-441b-8257-4871a18ed6ee","Type":"ContainerStarted","Data":"046f75a42df38c3e7006e037e67873e32671f79010604a38efc3cc8f7858d620"}
Oct 30 15:51:25.176199 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.176061    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:25.176199 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.176159    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:25.176304 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.176212    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:25.178100 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.178084    4585 generic.go:356] "Generic (PLEG): container finished" podID="8b400747-55f6-49cb-a8dc-3b547323787e" containerID="732a4ec34374b4b62e5d300dba5c04827d45f2656296acf47a34d626f789c7fb" exitCode=0
Oct 30 15:51:25.178228 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.178211    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" event={"ID":"8b400747-55f6-49cb-a8dc-3b547323787e","Type":"ContainerDied","Data":"732a4ec34374b4b62e5d300dba5c04827d45f2656296acf47a34d626f789c7fb"}
Oct 30 15:51:25.178926 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.178818    4585 scope.go:122] "RemoveContainer" containerID="732a4ec34374b4b62e5d300dba5c04827d45f2656296acf47a34d626f789c7fb"
Oct 30 15:51:25.184066 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.184017    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:25.184192 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.184112    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:25.184192 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.184161    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:25.698824 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.698791    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:25.880783 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.880743    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 15:51:25.880995 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:25.880798    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 15:51:26.197372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.194628    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" event={"ID":"ba4c3f49-4422-483f-b24f-41545ed46667","Type":"ContainerStarted","Data":"c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27"}
Oct 30 15:51:26.199207 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.198565    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" event={"ID":"8b400747-55f6-49cb-a8dc-3b547323787e","Type":"ContainerStarted","Data":"baa79aec8b0c8b4d61718a6e8387f84bceb42f676adc01b68a7eb144b2a805f6"}
Oct 30 15:51:26.294367 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.292925    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podStartSLOduration=2.292900067 podStartE2EDuration="2.292900067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:25.368995426 +0000 UTC m=+794.804599775" watchObservedRunningTime="2026-10-30 15:51:26.292900067 +0000 UTC m=+795.728504416"
Oct 30 15:51:26.603434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.603085    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:51:26.603434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.603261    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 15:51:26.603434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.603310    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 15:51:26.745376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.740866    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:51:26.745376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.742198    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 15:51:26.745376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.742266    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 15:51:26.881428 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.880913    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 15:51:26.881428 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:26.880989    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 15:51:27.590373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:27.589833    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 15:51:27.590373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:27.589910    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 15:51:27.740373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:27.739875    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 15:51:27.740373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:27.739953    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 15:51:27.883432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:27.881872    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 15:51:27.883432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:27.881945    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 15:51:28.339524 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:28.339468    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:51:28.590959 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:28.590855    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 15:51:28.591597 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:28.591548    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 15:51:28.639682 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:28.639641    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:51:28.642170 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:28.642148    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:51:28.755373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:28.754859    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 15:51:28.755373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:28.754918    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 15:51:28.883983 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:28.883883    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 15:51:28.884281 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:28.884220    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 15:51:29.589167 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:29.588730    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 15:51:29.589167 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:29.588791    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 15:51:29.647309 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:29.646446    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:51:29.739788 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:29.739755    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 15:51:29.740059 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:29.740029    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 15:51:29.902763 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:29.902725    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:51:29.910732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:29.910651    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:51:30.587687 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:30.587649    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 15:51:30.587886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:30.587803    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 15:51:30.742372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:30.740724    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 15:51:30.742372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:30.740788    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 15:51:31.588370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:31.587698    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 15:51:31.588370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:31.587759    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 15:51:31.764371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:31.763794    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 15:51:31.764371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:31.763856    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 15:51:32.591833 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:32.591800    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:51:32.592813 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:32.592791    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:51:32.746111 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:32.744564    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:51:32.748512 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:32.748483    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:51:34.701369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:34.701321    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:34.711492 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:34.711467    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:35.330951 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:35.330914    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:51:40.816290 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.816246    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"]
Oct 30 15:51:40.819897 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.819877    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:40.849001 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.848972    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"]
Oct 30 15:51:40.923255 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.923219    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:40.923469 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.923265    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:40.923469 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.923295    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75ecb5a4-e09c-4d59-b943-ea660626247c-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:40.923469 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.923431    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:40.923595 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.923471    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:40.923595 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.923567    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:40.923668 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.923593    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:40.923668 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.923661    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhg4l\" (UniqueName: \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-kube-api-access-qhg4l\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:40.923787 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:40.923714    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.024411 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.024379    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.024616 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.024441    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.024616 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.024479    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.024616 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.024513    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75ecb5a4-e09c-4d59-b943-ea660626247c-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.024616 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.024542    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.024616 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.024595    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.024891 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.024662    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.024891 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.024689    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.024891 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.024764    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhg4l\" (UniqueName: \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-kube-api-access-qhg4l\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.025039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.024985    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.025097 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.025082    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.025300 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.025277    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.025418 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.025310    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.025418 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.025402    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75ecb5a4-e09c-4d59-b943-ea660626247c-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.027150 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.027129    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.027567 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.027548    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.048692 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.048663    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.052385 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.052328    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhg4l\" (UniqueName: \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-kube-api-access-qhg4l\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.135548 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.135452    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:41.161931 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.161901    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"]
Oct 30 15:51:41.166461 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.166438    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.178313 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.178285    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"]
Oct 30 15:51:41.182774 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.182612    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.194164 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.189537    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"]
Oct 30 15:51:41.226432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.225811    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"]
Oct 30 15:51:41.226957 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.226927    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.227061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227016    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.227127 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227083    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.227127 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227118    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.227233 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227208    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9400972d-230d-4521-a683-bfc605f3b647-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.227382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227323    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.227448 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227408    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.227487 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227463    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.227521 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227490    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.227563 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227544    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.227628 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227569    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.227665 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227655    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.227767 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227751    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9400972d-230d-4521-a683-bfc605f3b647-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.227803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227783    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bdj2\" (UniqueName: \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-kube-api-access-2bdj2\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.227845 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227814    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.227881 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227847    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.227881 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227870    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.227950 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.227913    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlbjx\" (UniqueName: \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-kube-api-access-wlbjx\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.329253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329217    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9400972d-230d-4521-a683-bfc605f3b647-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.329253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329253    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bdj2\" (UniqueName: \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-kube-api-access-2bdj2\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.329532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329282    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.329532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329316    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.329532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329340    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.329532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329405    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlbjx\" (UniqueName: \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-kube-api-access-wlbjx\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.329532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329441    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.329532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329479    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.329532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329528    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.329893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329560    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.329893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329610    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9400972d-230d-4521-a683-bfc605f3b647-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.329893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329655    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.329893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329685    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.329893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329745    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.329893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329772    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.329893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329800    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.329893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329806    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.329893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329832    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.330365 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329896    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.330365 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329898    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.330365 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.329961    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.330365 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.330156    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9400972d-230d-4521-a683-bfc605f3b647-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.330562 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.330371    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.330562 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.330378    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.330672 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.330617    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.330672 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.330642    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.330852 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.330830    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.331244 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.331206    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.332427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.332403    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.332534 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.332497    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.332875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.332847    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.333039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.333023    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9400972d-230d-4521-a683-bfc605f3b647-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.352873 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.352847    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.353101 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.353078    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlbjx\" (UniqueName: \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-kube-api-access-wlbjx\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.353144 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.353091    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.353320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.353305    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bdj2\" (UniqueName: \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-kube-api-access-2bdj2\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.463529 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.463502    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"]
Oct 30 15:51:41.464039 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:51:41.464015    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ecb5a4_e09c_4d59_b943_ea660626247c.slice/crio-c65d729ec39b9542bb76cf12e900ac3175c342faca55c322e50fa4fdda3cf19e WatchSource:0}: Error finding container c65d729ec39b9542bb76cf12e900ac3175c342faca55c322e50fa4fdda3cf19e: Status 404 returned error can't find the container with id c65d729ec39b9542bb76cf12e900ac3175c342faca55c322e50fa4fdda3cf19e
Oct 30 15:51:41.466674 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.466637    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:41.466761 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.466703    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:41.466761 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.466732    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:41.484326 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.484299    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:41.510558 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.510533    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:41.809983 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.809954    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"]
Oct 30 15:51:41.810756 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:51:41.810731    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1d3da4_a476_4f37_b2c6_acc545bfd734.slice/crio-d01907a54e532cd67f4ca70c50f1a568895e6fbadcb2f45b2e75dd57f5b7a928 WatchSource:0}: Error finding container d01907a54e532cd67f4ca70c50f1a568895e6fbadcb2f45b2e75dd57f5b7a928: Status 404 returned error can't find the container with id d01907a54e532cd67f4ca70c50f1a568895e6fbadcb2f45b2e75dd57f5b7a928
Oct 30 15:51:41.813220 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.813189    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:41.813322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.813255    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:41.813322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.813283    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:41.850082 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.850048    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"]
Oct 30 15:51:41.851967 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:51:41.851928    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9400972d_230d_4521_a683_bfc605f3b647.slice/crio-4eed71234431aba7409af352151ac685d0aba5f25c48d8b83afba1e099c0d57d WatchSource:0}: Error finding container 4eed71234431aba7409af352151ac685d0aba5f25c48d8b83afba1e099c0d57d: Status 404 returned error can't find the container with id 4eed71234431aba7409af352151ac685d0aba5f25c48d8b83afba1e099c0d57d
Oct 30 15:51:41.855079 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.855037    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:41.855166 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.855126    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:41.855217 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:41.855174    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:42.383148 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.383021    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" event={"ID":"cd1d3da4-a476-4f37-b2c6-acc545bfd734","Type":"ContainerStarted","Data":"a3a54a84938ec4ee44391cb13e95ce96143d4059f245fb8858436c133faaf1fa"}
Oct 30 15:51:42.383148 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.383099    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" event={"ID":"cd1d3da4-a476-4f37-b2c6-acc545bfd734","Type":"ContainerStarted","Data":"d01907a54e532cd67f4ca70c50f1a568895e6fbadcb2f45b2e75dd57f5b7a928"}
Oct 30 15:51:42.385887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.385843    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" event={"ID":"9400972d-230d-4521-a683-bfc605f3b647","Type":"ContainerStarted","Data":"b376c7495326fa6c5273b37f2233d667c4436cebb002db40c05f17e7c26042bf"}
Oct 30 15:51:42.386016 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.385916    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" event={"ID":"9400972d-230d-4521-a683-bfc605f3b647","Type":"ContainerStarted","Data":"4eed71234431aba7409af352151ac685d0aba5f25c48d8b83afba1e099c0d57d"}
Oct 30 15:51:42.389081 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.389044    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" event={"ID":"75ecb5a4-e09c-4d59-b943-ea660626247c","Type":"ContainerStarted","Data":"1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3"}
Oct 30 15:51:42.389220 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.389106    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" event={"ID":"75ecb5a4-e09c-4d59-b943-ea660626247c","Type":"ContainerStarted","Data":"c65d729ec39b9542bb76cf12e900ac3175c342faca55c322e50fa4fdda3cf19e"}
Oct 30 15:51:42.484728 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.484697    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:42.487286 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.487253    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 15:51:42.487583 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.487546    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 15:51:42.499445 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.499399    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" podStartSLOduration=2.499383338 podStartE2EDuration="2.499383338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:42.498951049 +0000 UTC m=+811.934555398" watchObservedRunningTime="2026-10-30 15:51:42.499383338 +0000 UTC m=+811.934987685"
Oct 30 15:51:42.500712 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.500667    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" podStartSLOduration=1.500652569 podStartE2EDuration="1.500652569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:42.440039486 +0000 UTC m=+811.875643838" watchObservedRunningTime="2026-10-30 15:51:42.500652569 +0000 UTC m=+811.936256921"
Oct 30 15:51:42.511689 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.511571    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:42.513434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.513405    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 15:51:42.513671 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:42.513643    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 15:51:43.140374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:43.138522    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:43.140374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:43.138663    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 15:51:43.140374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:43.138709    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 15:51:43.486374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:43.484861    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 15:51:43.486374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:43.484926    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 15:51:43.512379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:43.511783    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 15:51:43.512379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:43.511845    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 15:51:44.138376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:44.137761    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 15:51:44.138376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:44.137825    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 15:51:44.526373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:44.525704    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 15:51:44.526373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:44.525758    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 15:51:44.526956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:44.526476    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 15:51:44.526956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:44.526515    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 15:51:45.139373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:45.138954    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 15:51:45.139373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:45.139021    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 15:51:45.486372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:45.485628    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 15:51:45.486372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:45.485704    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 15:51:45.512796 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:45.512760    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 15:51:45.513042 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:45.513010    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 15:51:46.136761 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:46.136724    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 15:51:46.137324 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:46.137293    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 15:51:46.488372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:46.484838    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 15:51:46.488372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:46.484918    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 15:51:46.516375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:46.514727    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 15:51:46.516375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:46.514788    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 15:51:47.144373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:47.144002    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 15:51:47.144373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:47.144073    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 15:51:47.496427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:47.495799    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 15:51:47.496427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:47.495864    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 15:51:47.522372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:47.514792    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 15:51:47.522372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:47.514854    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 15:51:48.104029 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:51:48.103950    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:51:48.137932 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:48.137882    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 15:51:48.138258 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:48.138211    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 15:51:48.485049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:48.485014    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 15:51:48.485703 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:48.485660    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 15:51:48.511988 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:48.511942    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Oct 30 15:51:48.512328 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:48.512293    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Oct 30 15:51:49.140209 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:49.140170    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:49.140637 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:49.140610    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:49.141258 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:49.141241    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:51:49.198514 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:49.198460    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" podStartSLOduration=8.198443987 podStartE2EDuration="8.198443987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:42.550923957 +0000 UTC m=+811.986528305" watchObservedRunningTime="2026-10-30 15:51:49.198443987 +0000 UTC m=+818.634048334"
Oct 30 15:51:49.491313 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:49.491283    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:49.491869 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:49.491709    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:49.492684 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:49.492658    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:51:49.515496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:49.515470    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:49.515848 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:49.515825    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:49.516717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:49.516700    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:51:56.234751 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.234715    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"]
Oct 30 15:51:56.239979 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.239931    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.272891 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.272853    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"]
Oct 30 15:51:56.434121 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.434075    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.434314 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.434206    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.434314 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.434247    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.434314 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.434269    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.434314 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.434298    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.434507 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.434379    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.434542 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.434524    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.434585 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.434567    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.434624 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.434588    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jc5j\" (UniqueName: \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-kube-api-access-7jc5j\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.536307 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536216    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.536307 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536278    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.536560 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536313    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.536560 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536332    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.536560 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536378    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.536793 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536767    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-credential-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.536897 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536807    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-socket\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.536897 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536828    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.536897 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536886    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-certs\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.537056 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536906    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.537056 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.536973    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.537056 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.537009    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jc5j\" (UniqueName: \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-kube-api-access-7jc5j\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.537203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.537055    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-data\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.537203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.537185    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istiod-ca-cert\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.538876 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.538852    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-envoy\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.539318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.539301    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-podinfo\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.560184 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.560152    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-token\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.560307 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.560202    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jc5j\" (UniqueName: \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-kube-api-access-7jc5j\") pod \"e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") " pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:56.857558 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:56.857468    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:57.179373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:57.179327    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"]
Oct 30 15:51:57.179703 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:51:57.179665    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db0f08f_678a_4f7e_99ee_8c7bb4de5af5.slice/crio-d7ae63ca5780745e73b7ba8c01496c44bb14064b4d58ba0aed1c19bf5e2cb873 WatchSource:0}: Error finding container d7ae63ca5780745e73b7ba8c01496c44bb14064b4d58ba0aed1c19bf5e2cb873: Status 404 returned error can't find the container with id d7ae63ca5780745e73b7ba8c01496c44bb14064b4d58ba0aed1c19bf5e2cb873
Oct 30 15:51:57.182088 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:57.182057    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:57.182210 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:57.182121    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:57.182210 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:57.182147    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:51:57.660913 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:57.660844    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" event={"ID":"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5","Type":"ContainerStarted","Data":"a8c23bb0aefb864400ec39cb9acdf63501934ca9e7472462eee2d2721861eafa"}
Oct 30 15:51:57.660913 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:57.660900    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" event={"ID":"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5","Type":"ContainerStarted","Data":"d7ae63ca5780745e73b7ba8c01496c44bb14064b4d58ba0aed1c19bf5e2cb873"}
Oct 30 15:51:57.720049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:57.719993    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" podStartSLOduration=1.719979737 podStartE2EDuration="1.719979737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:51:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:51:57.717330082 +0000 UTC m=+827.152934442" watchObservedRunningTime="2026-10-30 15:51:57.719979737 +0000 UTC m=+827.155584084"
Oct 30 15:51:57.858565 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:57.858521    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:51:58.862769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:58.862636    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 15:51:58.862769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:58.862713    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 15:51:59.859246 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:59.859204    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 15:51:59.859447 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:51:59.859270    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 15:52:00.858813 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:00.858771    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 15:52:00.859241 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:00.858844    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 15:52:01.858489 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:01.858454    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 15:52:01.858679 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:01.858521    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 15:52:02.858625 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:02.858589    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 15:52:02.859018 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:02.858644    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 15:52:03.839586 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.837391    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"]
Oct 30 15:52:03.841928 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.841902    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:03.863221 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.862783    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"]
Oct 30 15:52:03.874495 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.867516    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-ba9db9\"/\"premium-w5-ba9db9-kserve-self-signed-certs\""
Oct 30 15:52:03.874495 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.867870    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-ba9db9\"/\"kube-root-ca.crt\""
Oct 30 15:52:03.874495 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.868148    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-ba9db9\"/\"openshift-service-ca.crt\""
Oct 30 15:52:03.874495 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.871604    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-ba9db9\"/\"default-dockercfg-zrw7s\""
Oct 30 15:52:03.946041 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.946003    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-model-cache\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:03.946255 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.946051    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-tmp-dir\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:03.946255 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.946201    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb180805-8d51-47d0-8b08-4c26e39c8c50-tls-certs\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:03.946413 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.946391    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-dshm\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:03.946477 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.946450    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plqnb\" (UniqueName: \"kubernetes.io/projected/fb180805-8d51-47d0-8b08-4c26e39c8c50-kube-api-access-plqnb\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:03.946600 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:03.946581    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-home\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.048379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.048072    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-model-cache\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.048379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.048131    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-tmp-dir\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.048379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.048168    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb180805-8d51-47d0-8b08-4c26e39c8c50-tls-certs\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.048379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.048281    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-dshm\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.048379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.048327    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plqnb\" (UniqueName: \"kubernetes.io/projected/fb180805-8d51-47d0-8b08-4c26e39c8c50-kube-api-access-plqnb\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.048781 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.048439    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-home\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.052371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.048910    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-home\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.052371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.049157    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-model-cache\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.052371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.049388    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-tmp-dir\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.072101 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.072060    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb180805-8d51-47d0-8b08-4c26e39c8c50-tls-certs\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.081105 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.081071    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-dshm\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.094906 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.094819    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plqnb\" (UniqueName: \"kubernetes.io/projected/fb180805-8d51-47d0-8b08-4c26e39c8c50-kube-api-access-plqnb\") pod \"premium-w5-ba9db9-kserve-9dc89d6f8-km758\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") " pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.108889 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.106988    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:52:04.108889 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.108612    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:52:04.109906 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.109851    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:52:04.180936 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.180897    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:04.502796 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.502769    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"]
Oct 30 15:52:04.503260 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:52:04.503230    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb180805_8d51_47d0_8b08_4c26e39c8c50.slice/crio-8b130415345ab2cf622ea9461f2236983511a1b3b59e562d2d6e286ec836eb2a WatchSource:0}: Error finding container 8b130415345ab2cf622ea9461f2236983511a1b3b59e562d2d6e286ec836eb2a: Status 404 returned error can't find the container with id 8b130415345ab2cf622ea9461f2236983511a1b3b59e562d2d6e286ec836eb2a
Oct 30 15:52:04.705165 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.705128    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" event={"ID":"fb180805-8d51-47d0-8b08-4c26e39c8c50","Type":"ContainerStarted","Data":"c94338bb56283fa7176ab8799a224dbe42c898ce2eedf8a3f14ffa4a857ed05d"}
Oct 30 15:52:04.705323 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.705179    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" event={"ID":"fb180805-8d51-47d0-8b08-4c26e39c8c50","Type":"ContainerStarted","Data":"8b130415345ab2cf622ea9461f2236983511a1b3b59e562d2d6e286ec836eb2a"}
Oct 30 15:52:04.746638 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:04.746576    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podStartSLOduration=1.746558193 podStartE2EDuration="1.746558193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:52:04.746050838 +0000 UTC m=+834.181655187" watchObservedRunningTime="2026-10-30 15:52:04.746558193 +0000 UTC m=+834.182162542"
Oct 30 15:52:05.182014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:05.181981    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:14.181748 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:14.181708    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:14.197632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:14.197607    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:14.802003 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:14.801968    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:16.643794 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.643756    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"]
Oct 30 15:52:16.651368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.651319    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.663838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.663809    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"]
Oct 30 15:52:16.673127 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.673097    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-9e2b29\"/\"openshift-service-ca.crt\""
Oct 30 15:52:16.673305 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.673199    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-9e2b29\"/\"kube-root-ca.crt\""
Oct 30 15:52:16.673764 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.673737    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44baf777-0b73-472d-8c5f-cc7b693f0409-tls-certs\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.673919 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.673900    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-dshm\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.673984 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.673951    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-tmp-dir\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.674079 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.674060    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd55b\" (UniqueName: \"kubernetes.io/projected/44baf777-0b73-472d-8c5f-cc7b693f0409-kube-api-access-nd55b\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.674126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.674109    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-model-cache\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.674241 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.674225    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-home\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.678547 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.678523    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-9e2b29\"/\"default-dockercfg-8ttrr\""
Oct 30 15:52:16.678647 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.678540    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-9e2b29\"/\"premium-w2-9e2b29-kserve-self-signed-certs\""
Oct 30 15:52:16.775885 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.775849    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-home\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.776064 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.775902    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44baf777-0b73-472d-8c5f-cc7b693f0409-tls-certs\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.776064 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.775971    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-dshm\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.776064 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.775993    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-tmp-dir\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.776064 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.776051    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd55b\" (UniqueName: \"kubernetes.io/projected/44baf777-0b73-472d-8c5f-cc7b693f0409-kube-api-access-nd55b\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.776299 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.776094    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-model-cache\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.776400 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.776379    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-home\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.776496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.776470    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-tmp-dir\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.776547 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.776526    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-model-cache\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.778441 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.778417    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-dshm\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.778650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.778632    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44baf777-0b73-472d-8c5f-cc7b693f0409-tls-certs\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.799832 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.799804    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd55b\" (UniqueName: \"kubernetes.io/projected/44baf777-0b73-472d-8c5f-cc7b693f0409-kube-api-access-nd55b\") pod \"premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") " pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:16.969703 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:16.969668    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:17.273136 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:52:17.273107    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44baf777_0b73_472d_8c5f_cc7b693f0409.slice/crio-fad5a8e12f208524fc023d454b0568514faad13c7451f8d5b585254a8769429c WatchSource:0}: Error finding container fad5a8e12f208524fc023d454b0568514faad13c7451f8d5b585254a8769429c: Status 404 returned error can't find the container with id fad5a8e12f208524fc023d454b0568514faad13c7451f8d5b585254a8769429c
Oct 30 15:52:17.273256 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:17.273167    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"]
Oct 30 15:52:17.809237 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:17.809199    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" event={"ID":"44baf777-0b73-472d-8c5f-cc7b693f0409","Type":"ContainerStarted","Data":"1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae"}
Oct 30 15:52:17.809237 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:17.809240    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" event={"ID":"44baf777-0b73-472d-8c5f-cc7b693f0409","Type":"ContainerStarted","Data":"fad5a8e12f208524fc023d454b0568514faad13c7451f8d5b585254a8769429c"}
Oct 30 15:52:17.852037 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:17.851988    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podStartSLOduration=1.8519727480000001 podStartE2EDuration="1.851972748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:52:17.849865584 +0000 UTC m=+847.285469929" watchObservedRunningTime="2026-10-30 15:52:17.851972748 +0000 UTC m=+847.287577096"
Oct 30 15:52:17.964056 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:52:17.963964    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:52:17.970173 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:17.970145    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:19.880204 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.880169    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"]
Oct 30 15:52:19.884010 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.883988    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:19.900156 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.900129    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b24e7b\"/\"kube-root-ca.crt\""
Oct 30 15:52:19.912148 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.912120    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b24e7b\"/\"openshift-service-ca.crt\""
Oct 30 15:52:19.912406 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.912314    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b24e7b\"/\"premium-w4-b24e7b-kserve-self-signed-certs\""
Oct 30 15:52:19.912499 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.912439    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b24e7b\"/\"default-dockercfg-h2s6c\""
Oct 30 15:52:19.912813 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.912785    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-dshm\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:19.912903 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.912885    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7821afe-257e-4224-82f4-2b365bfe5dc6-tls-certs\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:19.913089 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.913067    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-home\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:19.913237 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.913213    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-tmp-dir\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:19.913339 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.913307    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmwhr\" (UniqueName: \"kubernetes.io/projected/e7821afe-257e-4224-82f4-2b365bfe5dc6-kube-api-access-kmwhr\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:19.913422 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.913363    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-model-cache\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:19.915095 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:19.915076    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"]
Oct 30 15:52:20.015046 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.015001    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-tmp-dir\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.015249 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.015081    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmwhr\" (UniqueName: \"kubernetes.io/projected/e7821afe-257e-4224-82f4-2b365bfe5dc6-kube-api-access-kmwhr\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.015249 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.015107    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-model-cache\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.015249 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.015219    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-dshm\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.015447 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.015316    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7821afe-257e-4224-82f4-2b365bfe5dc6-tls-certs\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.015447 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.015404    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-home\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.015566 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.015551    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-tmp-dir\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.015892 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.015866    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-home\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.016000 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.015912    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-model-cache\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.017766 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.017746    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-dshm\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.018173 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.018151    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7821afe-257e-4224-82f4-2b365bfe5dc6-tls-certs\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.039058 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.039030    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmwhr\" (UniqueName: \"kubernetes.io/projected/e7821afe-257e-4224-82f4-2b365bfe5dc6-kube-api-access-kmwhr\") pod \"premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") " pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.200722 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.200338    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:20.506102 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.506074    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"]
Oct 30 15:52:20.508355 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:52:20.508298    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7821afe_257e_4224_82f4_2b365bfe5dc6.slice/crio-754bf9bcc4db6a969845872606b52d4d94ea2d8f5f54793a719f984b39669af5 WatchSource:0}: Error finding container 754bf9bcc4db6a969845872606b52d4d94ea2d8f5f54793a719f984b39669af5: Status 404 returned error can't find the container with id 754bf9bcc4db6a969845872606b52d4d94ea2d8f5f54793a719f984b39669af5
Oct 30 15:52:20.828030 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.827935    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" event={"ID":"e7821afe-257e-4224-82f4-2b365bfe5dc6","Type":"ContainerStarted","Data":"ac9ae7a1fcc40b4a92ccb1810776255cf2d0aa65fad79ae78a282c13eed710d7"}
Oct 30 15:52:20.828030 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.827994    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" event={"ID":"e7821afe-257e-4224-82f4-2b365bfe5dc6","Type":"ContainerStarted","Data":"754bf9bcc4db6a969845872606b52d4d94ea2d8f5f54793a719f984b39669af5"}
Oct 30 15:52:20.872398 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:20.872317    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podStartSLOduration=1.872299033 podStartE2EDuration="1.872299033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:52:20.86849558 +0000 UTC m=+850.304099930" watchObservedRunningTime="2026-10-30 15:52:20.872299033 +0000 UTC m=+850.307903382"
Oct 30 15:52:21.201211 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:21.201171    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:25.965514 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:25.965418    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"]
Oct 30 15:52:25.985965 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:25.985934    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:25.993388 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:25.992671    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"]
Oct 30 15:52:26.130391 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.127189    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.130391 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.127296    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.130391 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.127333    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.145806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.135241    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.145806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.135287    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.145806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.135320    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l82bc\" (UniqueName: \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-kube-api-access-l82bc\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.145806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.135397    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.145806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.135431    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.145806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.135488    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.247252 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.247146    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.251147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.247341    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.251147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.247453    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.251147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.247484    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.251147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.247519    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.251147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.247545    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.251147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.247569    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l82bc\" (UniqueName: \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-kube-api-access-l82bc\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.251147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.247621    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.251147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.247654    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.251147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.250047    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-data\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.251988 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.251957    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-credential-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.252492 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.252471    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-socket\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.253014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.252988    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-certs\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.253638 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.253609    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istiod-ca-cert\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.303872 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.301385    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-envoy\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.303872 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.302176    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-podinfo\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.430407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.334178    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l82bc\" (UniqueName: \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-kube-api-access-l82bc\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.461380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.448156    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-token\") pod \"e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") " pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:26.649704 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:26.649665    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:27.083546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:27.004432    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:27.087163 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:27.086902    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:27.174380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:27.173900    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"]
Oct 30 15:52:27.224682 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:27.224623    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:52:27.224862 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:27.224733    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:52:27.224862 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:27.224780    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:52:27.921644 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:27.921600    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" event={"ID":"eafbf335-9c33-4b0b-92b6-da8f7b29d59f","Type":"ContainerStarted","Data":"a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed"}
Oct 30 15:52:27.921836 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:27.921665    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" event={"ID":"eafbf335-9c33-4b0b-92b6-da8f7b29d59f","Type":"ContainerStarted","Data":"b9def4c729e5a3be2d226dcda6505512fb9830fe909bb8b030c386b659477660"}
Oct 30 15:52:27.939640 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:27.935468    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:27.967769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:27.967700    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" podStartSLOduration=2.967683172 podStartE2EDuration="2.967683172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:52:27.966703687 +0000 UTC m=+857.402308035" watchObservedRunningTime="2026-10-30 15:52:27.967683172 +0000 UTC m=+857.403287521"
Oct 30 15:52:28.656544 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:28.656511    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:28.657618 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:28.657590    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 15:52:28.657824 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:28.657801    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 15:52:29.652839 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:29.652797    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 15:52:29.653130 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:29.653098    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 15:52:30.204829 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:30.204794    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:30.223371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:30.221099    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:30.653816 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:30.653766    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 15:52:30.654031 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:30.653832    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 15:52:30.973250 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:30.973177    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:31.653919 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:31.653877    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 15:52:31.654498 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:31.653935    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 15:52:32.653837 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:32.653790    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 15:52:32.654026 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:32.653872    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 15:52:33.652369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:33.650426    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 15:52:33.652369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:33.650505    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 15:52:34.661372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:34.660603    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:34.661894 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:34.661667    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:34.665337 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:34.665309    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:52:40.753022 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.752978    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"]
Oct 30 15:52:40.758511 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.758475    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.790966 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.790934    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-040f9a-openshift-default-dockercfg-r5zts\""
Oct 30 15:52:40.796390 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.795638    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"]
Oct 30 15:52:40.799967 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.799935    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-credential-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.800119 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.800053    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.800119 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.800100    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6ncj\" (UniqueName: \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-kube-api-access-q6ncj\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.800243 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.800185    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-data\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.800243 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.800222    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-token\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.800342 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.800319    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-envoy\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.800513 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.800494    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istiod-ca-cert\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.800580 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.800565    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-certs\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.800627 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.800594    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-podinfo\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.906171 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.906128    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-data\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.906470 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.906444    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-token\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.906704 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.906668    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-envoy\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.906977 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.906960    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istiod-ca-cert\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.907122 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.907107    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-certs\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.907229 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.907213    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-podinfo\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.907371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.907337    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-credential-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.907542 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.907526    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.907673 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.907656    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6ncj\" (UniqueName: \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-kube-api-access-q6ncj\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.908579 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.908553    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-data\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.925367 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.917018    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-podinfo\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.925367 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.917659    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istiod-ca-cert\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.925922 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.925896    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-envoy\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.927200 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.927175    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-credential-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.928639 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.928615    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-socket\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.928987 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.928969    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-certs\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.944931 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.944894    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-token\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:40.965189 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:40.965151    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6ncj\" (UniqueName: \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-kube-api-access-q6ncj\") pod \"e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") " pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:41.083077 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.077158    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:41.470558 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.470338    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"]
Oct 30 15:52:41.470719 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:52:41.470695    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0bee82_0d1d_4bb9_8575_3683fe99d5f2.slice/crio-44752e70f0f002574159318e10a3bd2164b5a55669850df47b7bda06d22bb811 WatchSource:0}: Error finding container 44752e70f0f002574159318e10a3bd2164b5a55669850df47b7bda06d22bb811: Status 404 returned error can't find the container with id 44752e70f0f002574159318e10a3bd2164b5a55669850df47b7bda06d22bb811
Oct 30 15:52:41.473637 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.473601    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:52:41.473751 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.473691    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:52:41.473751 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.473737    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:52:41.483925 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.483899    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"]
Oct 30 15:52:41.489225 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.489207    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.519467 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.519237    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"]
Oct 30 15:52:41.537869 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.537803    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-9e2b29-openshift-default-dockercfg-vfgdd\""
Oct 30 15:52:41.645372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.644103    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.645372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.644174    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a026ba91-cf65-4376-b0e2-386a4bd888ef-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.645372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.644232    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd87h\" (UniqueName: \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-kube-api-access-kd87h\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.645372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.644294    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.645372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.644338    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.645372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.644496    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.645372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.644529    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.645372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.644557    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.645372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.644602    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.748419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.747118    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.748419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.747183    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.748419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.747217    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.748419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.747261    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.748419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.747324    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.748419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.747379    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a026ba91-cf65-4376-b0e2-386a4bd888ef-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.748419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.747437    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd87h\" (UniqueName: \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-kube-api-access-kd87h\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.748419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.747504    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.748419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.747549    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.749009 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.748474    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.757371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.753708    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a026ba91-cf65-4376-b0e2-386a4bd888ef-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.757371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.754232    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.757371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.754504    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.757371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.754737    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.767370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.764970    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.773372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.769926    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.801370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.794122    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.801370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.796868    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd87h\" (UniqueName: \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-kube-api-access-kd87h\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:41.852371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:41.851588    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:42.126433 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:42.120183    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" event={"ID":"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2","Type":"ContainerStarted","Data":"70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce"}
Oct 30 15:52:42.126433 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:42.120255    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" event={"ID":"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2","Type":"ContainerStarted","Data":"44752e70f0f002574159318e10a3bd2164b5a55669850df47b7bda06d22bb811"}
Oct 30 15:52:42.237370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:42.235541    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" podStartSLOduration=2.235515005 podStartE2EDuration="2.235515005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:52:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:52:42.174976382 +0000 UTC m=+871.610580741" watchObservedRunningTime="2026-10-30 15:52:42.235515005 +0000 UTC m=+871.671119354"
Oct 30 15:52:42.877436 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:42.867773    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:52:42.988896 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:42.988851    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:52:43.022440 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.010418    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:52:43.101671 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.101637    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:43.120532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.120485    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:52:43.185404 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.176746    4585 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 15:52:43.185404 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.176808    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 15:52:43.324650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.293439    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:52:43.370968 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.367535    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"]
Oct 30 15:52:43.370968 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.370308    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:52:43.512496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.512435    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:52:43.512699 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.512547    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:52:43.512699 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.512600    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:52:43.875441 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.873964    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:52:43.878264 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:43.877131    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:44.004558 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.004497    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:52:44.014289 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.004825    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:44.073027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.072987    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:52:44.077363 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.076797    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:44.095438 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.094965    4585 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 15:52:44.095438 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.095021    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 15:52:44.121496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.121455    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:52:44.125427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.122173    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:52:44.430368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.408433    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:52:44.430368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.410688    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:52:44.430368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.412996    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:52:44.430368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.413146    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:52:44.685615 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.685083    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:52:44.791019 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.790979    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" event={"ID":"a026ba91-cf65-4376-b0e2-386a4bd888ef","Type":"ContainerStarted","Data":"7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502"}
Oct 30 15:52:44.791329 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.791305    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:52:44.793000 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.792976    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" event={"ID":"a026ba91-cf65-4376-b0e2-386a4bd888ef","Type":"ContainerStarted","Data":"6c4cc982e0a1860a1bc58dd236c4775622aa05b34f2e3bfba5f6f47fc15e3d42"}
Oct 30 15:52:44.799027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.799004    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:52:44.816375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.812719    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:52:44.816375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.813693    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:52:44.816375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.814741    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:52:44.868372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.858433    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:44.963428 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:44.962385    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" podStartSLOduration=3.962365402 podStartE2EDuration="3.962365402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:52:44.960118241 +0000 UTC m=+874.395722592" watchObservedRunningTime="2026-10-30 15:52:44.962365402 +0000 UTC m=+874.397969753"
Oct 30 15:52:45.081513 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:45.081153    4585 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 15:52:45.081513 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:45.081209    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 15:52:45.879751 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:45.876522    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 15:52:45.880150 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:45.880104    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 15:52:46.092374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:46.091850    4585 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 15:52:46.092374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:46.091909    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 15:52:46.865374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:46.858781    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 15:52:46.865374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:46.858845    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 15:52:47.079372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:47.078803    4585 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 15:52:47.079372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:47.078871    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 15:52:47.857373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:47.856779    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 15:52:47.857373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:47.856851    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 15:52:48.013159 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:52:48.013122    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:52:48.093416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:48.088717    4585 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 15:52:48.093416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:48.088786    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 15:52:48.857372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:48.856872    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 15:52:48.857372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:48.856948    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 15:52:49.081376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:49.078893    4585 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 15:52:49.081376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:49.078971    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 15:52:49.872376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:49.871951    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 15:52:49.872376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:49.872018    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 15:52:50.077994 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:50.077959    4585 patch_prober.go:29] interesting pod/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 15:52:50.078183 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:50.078024    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 15:52:50.856123 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:50.856080    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 15:52:50.856312 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:50.856152    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 15:52:51.082688 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:51.082652    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:51.083447 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:51.083426    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:51.084495 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:51.084473    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:52:51.860902 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:51.860874    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:51.861188 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:51.861165    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:51.861895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:51.861876    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:52:58.824373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:58.815825    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:52:58.824373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:58.816600    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:52:58.824373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:52:58.816896    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:53:07.854437 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:07.854398    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"]
Oct 30 15:53:07.858784 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:07.858759    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:07.879194 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:07.879161    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"]
Oct 30 15:53:07.886722 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:07.886688    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 15:53:07.886890 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:07.886766    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 15:53:07.989021 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:07.988984    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ede796e5-65bw4\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:07.989191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:07.989033    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ede796e5-65bw4\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:07.989191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:07.989129    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgr86\" (UniqueName: \"kubernetes.io/projected/c243a881-5d94-4c4c-a234-fd092457363b-kube-api-access-rgr86\") pod \"maas-api-revoke-sub-ede796e5-65bw4\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:08.090051 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:08.090003    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ede796e5-65bw4\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:08.090051 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:08.090063    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ede796e5-65bw4\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:08.090293 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:08.090111    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgr86\" (UniqueName: \"kubernetes.io/projected/c243a881-5d94-4c4c-a234-fd092457363b-kube-api-access-rgr86\") pod \"maas-api-revoke-sub-ede796e5-65bw4\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:08.091007 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:08.090967    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ede796e5-65bw4\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:08.092822 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:08.092793    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ede796e5-65bw4\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:08.113683 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:08.113620    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgr86\" (UniqueName: \"kubernetes.io/projected/c243a881-5d94-4c4c-a234-fd092457363b-kube-api-access-rgr86\") pod \"maas-api-revoke-sub-ede796e5-65bw4\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:08.171750 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:08.171720    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:08.649097 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:08.649063    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"]
Oct 30 15:53:08.663868 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:53:08.663827    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc243a881_5d94_4c4c_a234_fd092457363b.slice/crio-026a27b41b7b80fb99d061d9bb0b000977b509281cb045c867be04a3284224df WatchSource:0}: Error finding container 026a27b41b7b80fb99d061d9bb0b000977b509281cb045c867be04a3284224df: Status 404 returned error can't find the container with id 026a27b41b7b80fb99d061d9bb0b000977b509281cb045c867be04a3284224df
Oct 30 15:53:09.235743 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:09.235677    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4" event={"ID":"c243a881-5d94-4c4c-a234-fd092457363b","Type":"ContainerStarted","Data":"026a27b41b7b80fb99d061d9bb0b000977b509281cb045c867be04a3284224df"}
Oct 30 15:53:09.807372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:09.796449    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:53:09.807372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:09.806479    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:09.817371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:09.814630    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:53:09.817371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:09.815391    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:10.691538 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:10.691486    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:10.833228 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:10.832818    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:11.447561 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:11.447532    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:53:11.452133 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:11.452110    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:53:13.402823 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:13.402784    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4" event={"ID":"c243a881-5d94-4c4c-a234-fd092457363b","Type":"ContainerStarted","Data":"466c1143d3ca9ce8ba52f6acc78f845f8d53bb011cfb06f1eb7576405e0420f1"}
Oct 30 15:53:13.443188 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:13.443127    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4" podStartSLOduration=2.276796459 podStartE2EDuration="6.443108253s" totalImagesPullingTime="4.166311794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:53:13.442848188 +0000 UTC m=+902.878452549" watchObservedRunningTime="2026-10-30 15:53:13.443108253 +0000 UTC m=+902.878712601"
Oct 30 15:53:14.410445 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:14.410411    4585 generic.go:356] "Generic (PLEG): container finished" podID="c243a881-5d94-4c4c-a234-fd092457363b" containerID="466c1143d3ca9ce8ba52f6acc78f845f8d53bb011cfb06f1eb7576405e0420f1" exitCode=0
Oct 30 15:53:14.410906 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:14.410487    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4" event={"ID":"c243a881-5d94-4c4c-a234-fd092457363b","Type":"ContainerDied","Data":"466c1143d3ca9ce8ba52f6acc78f845f8d53bb011cfb06f1eb7576405e0420f1"}
Oct 30 15:53:15.572841 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:15.572815    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:15.654972 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:15.654936    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-openshift-ca\") pod \"c243a881-5d94-4c4c-a234-fd092457363b\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") "
Oct 30 15:53:15.655190 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:15.655066    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c243a881-5d94-4c4c-a234-fd092457363b-kube-api-access-rgr86\" (UniqueName: \"kubernetes.io/projected/c243a881-5d94-4c4c-a234-fd092457363b-kube-api-access-rgr86\") pod \"c243a881-5d94-4c4c-a234-fd092457363b\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") "
Oct 30 15:53:15.655190 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:15.655123    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-xks-ca\") pod \"c243a881-5d94-4c4c-a234-fd092457363b\" (UID: \"c243a881-5d94-4c4c-a234-fd092457363b\") "
Oct 30 15:53:15.655422 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:15.655381    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-openshift-ca" pod "c243a881-5d94-4c4c-a234-fd092457363b" (UID: "c243a881-5d94-4c4c-a234-fd092457363b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:53:15.655686 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:15.655659    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:53:15.657506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:15.657482    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-xks-ca" pod "c243a881-5d94-4c4c-a234-fd092457363b" (UID: "c243a881-5d94-4c4c-a234-fd092457363b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:53:15.657621 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:15.657531    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c243a881-5d94-4c4c-a234-fd092457363b-kube-api-access-rgr86" pod "c243a881-5d94-4c4c-a234-fd092457363b" (UID: "c243a881-5d94-4c4c-a234-fd092457363b"). InnerVolumeSpecName "kube-api-access-rgr86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:53:15.757315 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:15.757276    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgr86\" (UniqueName: \"kubernetes.io/projected/c243a881-5d94-4c4c-a234-fd092457363b-kube-api-access-rgr86\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:53:15.757315 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:15.757314    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c243a881-5d94-4c4c-a234-fd092457363b-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:53:16.424387 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:16.424343    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"
Oct 30 15:53:16.424581 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:16.424363    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4" event={"ID":"c243a881-5d94-4c4c-a234-fd092457363b","Type":"ContainerDied","Data":"026a27b41b7b80fb99d061d9bb0b000977b509281cb045c867be04a3284224df"}
Oct 30 15:53:16.424581 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:16.424490    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="026a27b41b7b80fb99d061d9bb0b000977b509281cb045c867be04a3284224df"
Oct 30 15:53:17.962529 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:53:17.962490    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:53:25.706372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:25.701477    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:25.857112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:25.856469    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:26.707502 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:26.707452    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:26.728775 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:26.728742    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:53:26.751750 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:26.751708    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:26.814509 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:26.814456    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:26.815178 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:26.814763    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:26.816377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:26.814795    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:26.877223 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:26.874455    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:53:26.877223 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:26.874906    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:53:27.727893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.726519    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:27.759370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.758864    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:27.767369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.761507    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:53:27.821384 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.816449    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:27.821384 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.816790    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:53:27.834408 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.825472    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:27.847429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.844130    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:53:27.847429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.844210    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:53:27.847429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.844251    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:53:27.878570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.878524    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:53:27.988440 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:27.985185    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": context deadline exceeded"
Oct 30 15:53:28.194275 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:28.194238    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:53:28.201189 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:28.199992    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:53:28.201957 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:28.201474    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:53:28.214095 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:28.214056    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:53:28.394440 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:28.393148    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:53:28.394440 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:28.393751    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:53:42.153901 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.153853    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"]
Oct 30 15:53:42.160467 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.160425    4585 cpu_manager.go:414] "containerMap: removing container" podUID="c243a881-5d94-4c4c-a234-fd092457363b" containerName="revoke-keys"
Oct 30 15:53:42.160689 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.160672    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c243a881-5d94-4c4c-a234-fd092457363b" containerName="revoke-keys"
Oct 30 15:53:42.160971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.160956    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="c243a881-5d94-4c4c-a234-fd092457363b" containerName="revoke-keys"
Oct 30 15:53:42.171549 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.171519    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.233372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.232405    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"]
Oct 30 15:53:42.307031 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.306994    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.307384 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.307362    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.307549 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.307531    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.307679 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.307665    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.307895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.307879    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtt9t\" (UniqueName: \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-kube-api-access-dtt9t\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.308114 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.308097    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.308249 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.308234    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.308462 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.308445    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.308648 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.308619    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.478850 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.478737    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.503533 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.503311    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.504245 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.504204    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.504448 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.503488    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.517653 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.517517    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.517962 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.517942    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.529541 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.529498    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtt9t\" (UniqueName: \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-kube-api-access-dtt9t\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.541696 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.530327    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.541972 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.541948    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.563374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.542700    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.564319 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.564287    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.564619 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.545326    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.564749 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.541652    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.565166 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.565145    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.691447 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.691382    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtt9t\" (UniqueName: \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-kube-api-access-dtt9t\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.692144 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.692092    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.696368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.692814    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:42.875463 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:42.875380    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:43.108618 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:43.108578    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:43.205329 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:43.205286    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:43.387499 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:43.387388    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:53:44.250649 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:44.250608    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"]
Oct 30 15:53:44.285306 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:44.285255    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:53:44.337371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:44.335462    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:53:44.337371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:44.335538    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:53:44.892431 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:44.892165    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" event={"ID":"fcb5f24d-b29c-4940-99cc-197fd3fde8ca","Type":"ContainerStarted","Data":"517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b"}
Oct 30 15:53:44.892431 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:44.892272    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" event={"ID":"fcb5f24d-b29c-4940-99cc-197fd3fde8ca","Type":"ContainerStarted","Data":"66532253f74eaf335306810e4df25d45ce44f34edb26050caa2b639a9ed3fdcc"}
Oct 30 15:53:44.971289 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:44.971213    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" podStartSLOduration=2.971190953 podStartE2EDuration="2.971190953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:53:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:53:44.952906806 +0000 UTC m=+934.388511154" watchObservedRunningTime="2026-10-30 15:53:44.971190953 +0000 UTC m=+934.406795304"
Oct 30 15:53:45.109889 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:45.109801    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:45.111112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:45.111076    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 15:53:45.111262 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:45.111133    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 15:53:46.109258 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:46.109225    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 15:53:46.109709 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:46.109277    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 15:53:47.109788 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:47.109756    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 15:53:47.110215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:47.109811    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 15:53:47.986548 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:53:47.986510    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:53:48.109969 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:48.109934    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 15:53:48.110428 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:48.109995    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 15:53:49.109285 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:49.109249    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 15:53:49.109514 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:49.109314    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 15:53:50.110038 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:50.109995    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 15:53:50.110537 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:50.110068    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 15:53:51.109635 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:51.109597    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 15:53:51.109846 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:51.109661    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 15:53:52.285292 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.283934    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:52.285292 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.285235    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:52.285986 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.285965    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:53:52.747141 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.747106    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"]
Oct 30 15:53:52.751441 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.751421    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.781080 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.781042    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"]
Oct 30 15:53:52.857616 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.857577    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.857616 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.857617    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.857904 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.857637    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.857904 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.857654    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.857904 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.857794    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8mxd\" (UniqueName: \"kubernetes.io/projected/619e7d6d-2940-4118-81a9-13a16430f65b-kube-api-access-x8mxd\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.857904 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.857852    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/619e7d6d-2940-4118-81a9-13a16430f65b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.858122 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.857982    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/619e7d6d-2940-4118-81a9-13a16430f65b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.858122 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.858060    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/619e7d6d-2940-4118-81a9-13a16430f65b-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.858194 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.858173    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.959811 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.959777    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960020 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.959855    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960020 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.959889    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960020 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.959947    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960020 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.959975    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960243 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.960025    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8mxd\" (UniqueName: \"kubernetes.io/projected/619e7d6d-2940-4118-81a9-13a16430f65b-kube-api-access-x8mxd\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960243 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.960124    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/619e7d6d-2940-4118-81a9-13a16430f65b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.960245    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/619e7d6d-2940-4118-81a9-13a16430f65b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.960282    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-workload-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.960336    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-workload-certs\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960537 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.960380    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/619e7d6d-2940-4118-81a9-13a16430f65b-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.960848 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.960819    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-istio-data\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.961052 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.961032    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/619e7d6d-2940-4118-81a9-13a16430f65b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.961136 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.961033    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-credential-socket\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.962266 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.962243    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/619e7d6d-2940-4118-81a9-13a16430f65b-istio-envoy\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.962764 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.962742    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/619e7d6d-2940-4118-81a9-13a16430f65b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.986032 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.985993    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/619e7d6d-2940-4118-81a9-13a16430f65b-istio-token\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:52.986568 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:52.986545    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8mxd\" (UniqueName: \"kubernetes.io/projected/619e7d6d-2940-4118-81a9-13a16430f65b-kube-api-access-x8mxd\") pod \"maas-default-gateway-openshift-default-9dccdc9b7-rtvxh\" (UID: \"619e7d6d-2940-4118-81a9-13a16430f65b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:53.067108 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:53.067024    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:53.391443 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:53.391412    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"]
Oct 30 15:53:53.392257 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:53:53.392231    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod619e7d6d_2940_4118_81a9_13a16430f65b.slice/crio-e3d3e8489b74ab0dc66f97f3afd9df9a52981bd689ad03d1c99f6bd892092600 WatchSource:0}: Error finding container e3d3e8489b74ab0dc66f97f3afd9df9a52981bd689ad03d1c99f6bd892092600: Status 404 returned error can't find the container with id e3d3e8489b74ab0dc66f97f3afd9df9a52981bd689ad03d1c99f6bd892092600
Oct 30 15:53:53.394748 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:53.394712    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:53:53.394823 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:53.394780    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:53:53.394823 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:53.394808    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:53:53.988678 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:53.988598    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" event={"ID":"619e7d6d-2940-4118-81a9-13a16430f65b","Type":"ContainerStarted","Data":"2be194376a5270cc60ea2aa72cc2b6b37f5f9f9e93f288ca5b6e86817ec2b248"}
Oct 30 15:53:53.988678 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:53.988666    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" event={"ID":"619e7d6d-2940-4118-81a9-13a16430f65b","Type":"ContainerStarted","Data":"e3d3e8489b74ab0dc66f97f3afd9df9a52981bd689ad03d1c99f6bd892092600"}
Oct 30 15:53:54.039871 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:54.039809    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" podStartSLOduration=2.039792399 podStartE2EDuration="2.039792399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:53:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:53:54.037414236 +0000 UTC m=+943.473018585" watchObservedRunningTime="2026-10-30 15:53:54.039792399 +0000 UTC m=+943.475396746"
Oct 30 15:53:54.067782 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:54.067747    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:53:54.069536 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:54.069510    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 15:53:54.069677 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:54.069562    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" podUID="619e7d6d-2940-4118-81a9-13a16430f65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 15:53:55.068009 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:55.067971    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 15:53:55.068635 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:55.068594    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" podUID="619e7d6d-2940-4118-81a9-13a16430f65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 15:53:56.067991 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:56.067951    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 15:53:56.068206 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:56.068017    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" podUID="619e7d6d-2940-4118-81a9-13a16430f65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 15:53:57.067671 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:57.067637    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 15:53:57.067843 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:57.067702    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" podUID="619e7d6d-2940-4118-81a9-13a16430f65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 15:53:58.070375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:58.069779    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 15:53:58.070375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:58.069842    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" podUID="619e7d6d-2940-4118-81a9-13a16430f65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 15:53:59.071800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:59.071755    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 15:53:59.072423 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:53:59.072387    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" podUID="619e7d6d-2940-4118-81a9-13a16430f65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 15:54:00.072373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:00.070455    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 15:54:00.072373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:00.070527    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" podUID="619e7d6d-2940-4118-81a9-13a16430f65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 15:54:01.068326 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:01.068288    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 15:54:01.068521 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:01.068380    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" podUID="619e7d6d-2940-4118-81a9-13a16430f65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 15:54:02.664799 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:02.664759    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:54:02.665395 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:02.665140    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:54:02.666369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:02.666333    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh"
Oct 30 15:54:10.348385 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.348320    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"]
Oct 30 15:54:10.352814 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.352789    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.372205 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.372175    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"]
Oct 30 15:54:10.379583 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.379555    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-c129d1\"/\"distinct-2-w6-c129d1-kserve-self-signed-certs\""
Oct 30 15:54:10.379750 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.379613    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-c129d1\"/\"default-dockercfg-kc6fq\""
Oct 30 15:54:10.381249 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.381229    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-c129d1\"/\"openshift-service-ca.crt\""
Oct 30 15:54:10.381392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.381255    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-c129d1\"/\"kube-root-ca.crt\""
Oct 30 15:54:10.421913 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.421873    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w4hj\" (UniqueName: \"kubernetes.io/projected/4067e045-ff27-4da7-99e3-e6e6026b182e-kube-api-access-5w4hj\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.422068 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.422015    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-home\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.422178 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.422157    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4067e045-ff27-4da7-99e3-e6e6026b182e-tls-certs\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.422247 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.422231    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-model-cache\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.422363 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.422334    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-dshm\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.422424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.422409    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-tmp-dir\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.523790 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.523748    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w4hj\" (UniqueName: \"kubernetes.io/projected/4067e045-ff27-4da7-99e3-e6e6026b182e-kube-api-access-5w4hj\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.523977 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.523864    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-home\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.523977 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.523933    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4067e045-ff27-4da7-99e3-e6e6026b182e-tls-certs\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.524100 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.523992    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-model-cache\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.524100 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.524068    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-dshm\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.524205 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.524100    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-tmp-dir\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.524377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.524337    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-home\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.524486 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.524379    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-model-cache\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.524486 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.524463    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-tmp-dir\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.526235 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.526210    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-dshm\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.526497 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.526477    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4067e045-ff27-4da7-99e3-e6e6026b182e-tls-certs\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.548128 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.548095    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w4hj\" (UniqueName: \"kubernetes.io/projected/4067e045-ff27-4da7-99e3-e6e6026b182e-kube-api-access-5w4hj\") pod \"distinct-2-w6-c129d1-kserve-784b6476b6-5gttl\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") " pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:10.678898 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.678863    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:11.000482 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:10.988832    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"]
Oct 30 15:54:11.120389 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:11.120334    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" event={"ID":"4067e045-ff27-4da7-99e3-e6e6026b182e","Type":"ContainerStarted","Data":"f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116"}
Oct 30 15:54:11.120554 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:11.120401    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" event={"ID":"4067e045-ff27-4da7-99e3-e6e6026b182e","Type":"ContainerStarted","Data":"f811cddde00b1e668783c20490735ef6c31c792d6e34911f4d248b7c2f0827d8"}
Oct 30 15:54:11.164104 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:11.164053    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podStartSLOduration=1.164038572 podStartE2EDuration="1.164038572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:54:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:54:11.161296064 +0000 UTC m=+960.596900415" watchObservedRunningTime="2026-10-30 15:54:11.164038572 +0000 UTC m=+960.599642920"
Oct 30 15:54:11.679593 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:11.679550    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:17.965499 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:54:17.965327    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:54:20.679371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.679316    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:20.692525 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.692499    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:20.778264 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.778214    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"]
Oct 30 15:54:20.788672 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.788639    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.804901 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.804869    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"]
Oct 30 15:54:20.806146 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.805433    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b63c3c\"/\"distinct-2-w0-b63c3c-kserve-self-signed-certs\""
Oct 30 15:54:20.868500 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.868467    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvzjn\" (UniqueName: \"kubernetes.io/projected/dfa4b03a-3d65-4fdf-99cc-704df480ca67-kube-api-access-gvzjn\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.868715 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.868596    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-model-cache\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.868715 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.868694    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-home\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.868859 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.868839    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-dshm\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.868911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.868894    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tmp-dir\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.868984 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.868959    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tls-certs\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.985215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.980569    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-dshm\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.985215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.980644    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tmp-dir\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.985215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.980673    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tls-certs\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.985215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.980776    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvzjn\" (UniqueName: \"kubernetes.io/projected/dfa4b03a-3d65-4fdf-99cc-704df480ca67-kube-api-access-gvzjn\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.985215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.980835    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-model-cache\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.985215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.980902    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-home\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.985215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.981445    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-home\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.985215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.983834    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-model-cache\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.985215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.984127    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tmp-dir\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.985800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.984707    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-dshm\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:20.989367 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:20.985930    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tls-certs\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:21.008034 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:21.007974    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvzjn\" (UniqueName: \"kubernetes.io/projected/dfa4b03a-3d65-4fdf-99cc-704df480ca67-kube-api-access-gvzjn\") pod \"distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") " pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:21.111480 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:21.111444    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:21.202102 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:21.201896    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:54:21.428757 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:54:21.428722    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfa4b03a_3d65_4fdf_99cc_704df480ca67.slice/crio-215714ec4db993ef1981dbae6d3178ae4a5a1c062a0f438697cb961b33cb9e94 WatchSource:0}: Error finding container 215714ec4db993ef1981dbae6d3178ae4a5a1c062a0f438697cb961b33cb9e94: Status 404 returned error can't find the container with id 215714ec4db993ef1981dbae6d3178ae4a5a1c062a0f438697cb961b33cb9e94
Oct 30 15:54:21.428990 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:21.428963    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"]
Oct 30 15:54:22.194827 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:22.194786    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" event={"ID":"dfa4b03a-3d65-4fdf-99cc-704df480ca67","Type":"ContainerStarted","Data":"0593593208e1643e0a0a94cf0093456b9862fec39064160d258338d867d6edd9"}
Oct 30 15:54:22.195274 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:22.194838    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" event={"ID":"dfa4b03a-3d65-4fdf-99cc-704df480ca67","Type":"ContainerStarted","Data":"215714ec4db993ef1981dbae6d3178ae4a5a1c062a0f438697cb961b33cb9e94"}
Oct 30 15:54:22.236869 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:22.236807    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podStartSLOduration=2.236786441 podStartE2EDuration="2.236786441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:54:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:54:22.23652948 +0000 UTC m=+971.672133831" watchObservedRunningTime="2026-10-30 15:54:22.236786441 +0000 UTC m=+971.672390790"
Oct 30 15:54:23.111944 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:23.111907    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:31.112393 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:31.112334    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:31.125231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:31.125206    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:31.282126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:31.282098    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:54:42.290751 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.290717    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"]
Oct 30 15:54:42.294981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.294957    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.331861 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.331803    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"]
Oct 30 15:54:42.337152 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.337118    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"]
Oct 30 15:54:42.337298 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.337262    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.359535 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.359505    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"]
Oct 30 15:54:42.438638 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.438603    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.438638 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.438642    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.438879 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.438661    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.438879 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.438864    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.438982 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.438892    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.438982 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.438919    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.438982 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.438962    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.439147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439102    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsbwq\" (UniqueName: \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-kube-api-access-vsbwq\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.439211 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439194    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.439269 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439249    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.439321 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439275    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00af3987-7922-49e4-a15f-9d4caeaa813d-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.439408 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439394    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r54qs\" (UniqueName: \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-kube-api-access-r54qs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.439485 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439425    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.439485 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439451    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.439563 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439506    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.439563 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439536    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.439643 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439616    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.439677 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.439648    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.540698 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.540658    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsbwq\" (UniqueName: \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-kube-api-access-vsbwq\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.540934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.540726    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.540934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.540764    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.540934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.540797    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00af3987-7922-49e4-a15f-9d4caeaa813d-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.540934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.540851    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r54qs\" (UniqueName: \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-kube-api-access-r54qs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.540934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.540871    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.540934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.540887    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.540934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.540909    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.541416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.540944    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.541416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541007    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.541416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541052    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.541416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541095    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.541416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541125    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.541416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541152    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.541416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541143    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.541416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541279    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.541416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541379    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-credential-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.541886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541444    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.541886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541528    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-data\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.541886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541668    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-socket\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.541886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541773    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.541886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541781    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00af3987-7922-49e4-a15f-9d4caeaa813d-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.541886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541808    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.541886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541846    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.541886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.541882    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.542327 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.542188    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istiod-ca-cert\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.542327 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.542261    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.542327 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.542312    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-certs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.543920 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.543889    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.543997 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.543936    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-podinfo\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.544224 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.544204    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.544537 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.544520    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-envoy\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.564062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.564029    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.564297 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.564275    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r54qs\" (UniqueName: \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-kube-api-access-r54qs\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.564398 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.564343    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsbwq\" (UniqueName: \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-kube-api-access-vsbwq\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.564754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.564737    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-token\") pod \"e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") " pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.611126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.611083    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:42.656702 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.656670    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:42.942628 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.942598    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"]
Oct 30 15:54:42.943142 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:54:42.943118    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6c8cdc3_f4e8_4a20_9015_423675fe5e55.slice/crio-283dc77340ede50bfa6e9cfd7903ce3f3c7f28668ee380d11e07b65c414386c9 WatchSource:0}: Error finding container 283dc77340ede50bfa6e9cfd7903ce3f3c7f28668ee380d11e07b65c414386c9: Status 404 returned error can't find the container with id 283dc77340ede50bfa6e9cfd7903ce3f3c7f28668ee380d11e07b65c414386c9
Oct 30 15:54:42.945605 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.945570    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:54:42.945696 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.945650    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:54:42.945765 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.945693    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:54:42.990216 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.990185    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"]
Oct 30 15:54:42.991028 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:54:42.991000    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00af3987_7922_49e4_a15f_9d4caeaa813d.slice/crio-80fba14c56196a97c530ca3c3c48f30a89efb58aa10ab959ce48b4be3c8f94a5 WatchSource:0}: Error finding container 80fba14c56196a97c530ca3c3c48f30a89efb58aa10ab959ce48b4be3c8f94a5: Status 404 returned error can't find the container with id 80fba14c56196a97c530ca3c3c48f30a89efb58aa10ab959ce48b4be3c8f94a5
Oct 30 15:54:42.994410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.994374    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:54:42.994479 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.994463    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:54:42.994522 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:42.994506    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:54:43.384338 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:43.384233    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" event={"ID":"00af3987-7922-49e4-a15f-9d4caeaa813d","Type":"ContainerStarted","Data":"76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad"}
Oct 30 15:54:43.384338 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:43.384307    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" event={"ID":"00af3987-7922-49e4-a15f-9d4caeaa813d","Type":"ContainerStarted","Data":"80fba14c56196a97c530ca3c3c48f30a89efb58aa10ab959ce48b4be3c8f94a5"}
Oct 30 15:54:43.386408 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:43.386378    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" event={"ID":"e6c8cdc3-f4e8-4a20-9015-423675fe5e55","Type":"ContainerStarted","Data":"cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194"}
Oct 30 15:54:43.386570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:43.386436    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" event={"ID":"e6c8cdc3-f4e8-4a20-9015-423675fe5e55","Type":"ContainerStarted","Data":"283dc77340ede50bfa6e9cfd7903ce3f3c7f28668ee380d11e07b65c414386c9"}
Oct 30 15:54:43.432005 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:43.431941    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" podStartSLOduration=1.431920176 podStartE2EDuration="1.431920176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:54:43.430583001 +0000 UTC m=+992.866187418" watchObservedRunningTime="2026-10-30 15:54:43.431920176 +0000 UTC m=+992.867524521"
Oct 30 15:54:43.480462 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:43.480406    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" podStartSLOduration=1.480390152 podStartE2EDuration="1.480390152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:54:43.478664104 +0000 UTC m=+992.914268453" watchObservedRunningTime="2026-10-30 15:54:43.480390152 +0000 UTC m=+992.915994501"
Oct 30 15:54:43.611918 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:43.611877    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:43.660369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:43.659447    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:44.612538 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:44.612504    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 15:54:44.612935 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:44.612558    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 15:54:44.691398 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:44.674855    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 15:54:44.691398 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:44.674916    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 15:54:45.611827 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:45.611790    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 15:54:45.612015 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:45.611854    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 15:54:45.658370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:45.657876    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 15:54:45.658370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:45.657963    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 15:54:46.612456 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:46.612422    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 15:54:46.612636 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:46.612492    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 15:54:46.685518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:46.672818    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 15:54:46.685518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:46.672893    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 15:54:47.612372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:47.612317    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 15:54:47.612566 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:47.612398    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 15:54:47.658370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:47.657766    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 15:54:47.658370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:47.657826    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 15:54:48.012396 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:54:48.012332    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:54:48.612310 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:48.612273    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 15:54:48.612493 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:48.612331    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 15:54:48.663372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:48.663134    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 15:54:48.663372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:48.663211    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 15:54:49.611756 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:49.611716    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 15:54:49.612574 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:49.611779    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 15:54:49.657822 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:49.657778    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 15:54:49.658038 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:49.657848    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 15:54:50.739369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:50.735529    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:50.739369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:50.736713    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:50.739369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:50.737500    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:54:50.986005 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:50.985971    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:50.986461 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:50.986395    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:54:50.987490 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:54:50.987463    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:55:10.192481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:10.192433    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:55:10.201335 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:10.201294    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:10.202011 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:10.201980    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:10.219479 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:10.219431    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:10.288697 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:10.287716    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:10.395493 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:10.395450    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:10.732477 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:10.730452    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:11.230687 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.230647    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:11.410485 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.410434    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:55:11.436504 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.277567    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:11.439454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.439400    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:55:11.446580 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.277638    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:55:11.466376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.318827    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:55:11.466376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.448532    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:55:11.466376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.448591    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:55:11.466376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.230416    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:11.466376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.412297    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:55:11.466376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.277615    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:55:11.466376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.452259    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:55:11.466376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.452313    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:55:11.563246 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.563155    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:55:11.737724 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.737663    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:11.739451 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.738959    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": context deadline exceeded"
Oct 30 15:55:11.822338 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:11.822236    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:55:12.025438 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.025393    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:55:12.180373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.158129    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:12.412379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.409667    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:55:12.435627 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.423904    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:55:12.435627 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.434401    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:55:12.435627 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.434464    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:55:12.466608 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.465273    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:55:12.467497 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.467445    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:12.473981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.473240    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:55:12.515207 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.515180    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:55:12.725958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.725855    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"]
Oct 30 15:55:12.731964 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.731927    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"]
Oct 30 15:55:12.731964 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.731970    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"]
Oct 30 15:55:12.732183 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.732049    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.736147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.736125    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:12.737727 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.737706    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"]
Oct 30 15:55:12.827805 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.827763    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.827956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.827917    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.828029 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.827953    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64rxm\" (UniqueName: \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-kube-api-access-64rxm\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.828223 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.828204    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.828309 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.828242    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.828384 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.828366    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.828632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.828613    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.828716 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.828646    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.828791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.828774    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.930742 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.930702    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.930945 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.930751    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64rxm\" (UniqueName: \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-kube-api-access-64rxm\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.930945 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.930904    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:12.931082 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.930952    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:12.931082 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.931044    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:12.931194 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.931150    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.931194 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.931163    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.931194 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.931184    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:12.931329 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.931298    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.931424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.931338    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.931424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.931398    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:12.931533 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.931449    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.931533 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.931488    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.931533 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.931512    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.932265 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.932242    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.932444 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.932421    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.932529 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.932476    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.932529 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.932501    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:12.932529 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.932517    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb4s7\" (UniqueName: \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-kube-api-access-vb4s7\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:12.932685 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.932539    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:12.932685 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.932590    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:12.932893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.932868    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.932950 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.932877    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.934295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.934259    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.934532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.934515    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.954708 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.954673    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64rxm\" (UniqueName: \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-kube-api-access-64rxm\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:12.955339 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:12.955317    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:13.034412 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034310    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.034412 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034372    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.034639 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034539    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.034639 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034562    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb4s7\" (UniqueName: \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-kube-api-access-vb4s7\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.034639 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034583    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.034639 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034634    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.034875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034779    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.034875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034829    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.034991 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034861    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.034991 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034958    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-certs\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.035105 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.034996    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-data\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.035274 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.035252    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-credential-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.035375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.035269    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istiod-ca-cert\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.035439 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.035416    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-socket\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.036930 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.036909    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-podinfo\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.037175 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.037153    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-envoy\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.056006 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.054252    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:13.058838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.058778    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-token\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.061069 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.060663    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb4s7\" (UniqueName: \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-kube-api-access-vb4s7\") pod \"e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") " pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.061069 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.060870    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:13.398191 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:55:13.398122    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad89200_6a9a_4884_a94c_8056a9ebeb49.slice/crio-9b596aeba57097e8b1b49aebfea9e660383f0706009de3045c7f64cef076bc99 WatchSource:0}: Error finding container 9b596aeba57097e8b1b49aebfea9e660383f0706009de3045c7f64cef076bc99: Status 404 returned error can't find the container with id 9b596aeba57097e8b1b49aebfea9e660383f0706009de3045c7f64cef076bc99
Oct 30 15:55:13.399247 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.399223    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"]
Oct 30 15:55:13.400925 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.400892    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:55:13.401022 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.400964    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:55:13.401022 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.401004    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:55:13.401117 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:55:13.401046    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91cc851d_7402_44c4_9f5c_88c5a5c6a90c.slice/crio-3fc5ef7c7ac24235311c4d9d1141009e78e1bd0753052cf4b1167885d72a77ca WatchSource:0}: Error finding container 3fc5ef7c7ac24235311c4d9d1141009e78e1bd0753052cf4b1167885d72a77ca: Status 404 returned error can't find the container with id 3fc5ef7c7ac24235311c4d9d1141009e78e1bd0753052cf4b1167885d72a77ca
Oct 30 15:55:13.402419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.402401    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"]
Oct 30 15:55:13.403510 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.403443    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:55:13.403510 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.403495    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:55:13.403623 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:13.403539    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:55:14.305968 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:14.305927    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" event={"ID":"91cc851d-7402-44c4-9f5c-88c5a5c6a90c","Type":"ContainerStarted","Data":"3ce422b3b9516f9c8a957e595cd590864e6f3582467cce2c56fa21f661cc3999"}
Oct 30 15:55:14.306493 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:14.306021    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" event={"ID":"91cc851d-7402-44c4-9f5c-88c5a5c6a90c","Type":"ContainerStarted","Data":"3fc5ef7c7ac24235311c4d9d1141009e78e1bd0753052cf4b1167885d72a77ca"}
Oct 30 15:55:14.309047 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:14.309012    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" event={"ID":"0ad89200-6a9a-4884-a94c-8056a9ebeb49","Type":"ContainerStarted","Data":"87c56bfe4734c59c4bed9eaf4c096fd8a6aa869743b96fffe2fd46c7e67f4818"}
Oct 30 15:55:14.309176 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:14.309074    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" event={"ID":"0ad89200-6a9a-4884-a94c-8056a9ebeb49","Type":"ContainerStarted","Data":"9b596aeba57097e8b1b49aebfea9e660383f0706009de3045c7f64cef076bc99"}
Oct 30 15:55:14.388828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:14.388760    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" podStartSLOduration=2.38874003 podStartE2EDuration="2.38874003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:55:14.386867669 +0000 UTC m=+1023.822472017" watchObservedRunningTime="2026-10-30 15:55:14.38874003 +0000 UTC m=+1023.824344378"
Oct 30 15:55:14.455372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:14.436144    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podStartSLOduration=2.436125059 podStartE2EDuration="2.436125059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:55:14.435267622 +0000 UTC m=+1023.870871970" watchObservedRunningTime="2026-10-30 15:55:14.436125059 +0000 UTC m=+1023.871729411"
Oct 30 15:55:15.061558 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:15.061522    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:15.063378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:15.063337    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:15.063597 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:15.063254    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 15:55:15.063765 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:15.063741    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 15:55:15.064229 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:15.064202    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 15:55:15.064408 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:15.064386    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 15:55:16.058373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:16.057786    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 15:55:16.058373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:16.057868    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 15:55:16.081113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:16.064737    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 15:55:16.081113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:16.064791    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 15:55:17.061372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:17.059761    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 15:55:17.061372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:17.059824    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 15:55:17.066377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:17.063717    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 15:55:17.066377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:17.063771    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 15:55:18.065851 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:18.065810    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 15:55:18.066504 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:18.066467    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 15:55:18.067377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:18.067338    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 15:55:18.067535 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:18.067516    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 15:55:18.116156 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:55:18.116089    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:55:19.062820 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:19.062780    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 15:55:19.063109 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:19.063079    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 15:55:19.067780 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:19.067742    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 15:55:19.068339 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:19.068307    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 15:55:20.056744 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:20.056698    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 15:55:20.056935 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:20.056790    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 15:55:20.062287 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:20.062243    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 15:55:20.062488 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:20.062310    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 15:55:21.076143 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:21.069068    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:21.076143 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:21.071255    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:21.076143 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:21.072463    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:55:21.122760 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:21.122729    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:21.125041 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:21.125013    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:21.132605 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:21.128748    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:55:26.447441 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:26.446443    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:26.447969 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:26.447471    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:26.479896 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:26.479015    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:26.517509 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:26.517461    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:26.567476 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:26.567424    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:55:26.890721 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:26.890576    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:27.449528 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:27.449462    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:27.461373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:27.452317    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:55:27.486587 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:27.486507    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:27.487298 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:27.487277    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:55:27.527333 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:27.527293    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:27.528203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:27.528179    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:55:27.592230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:27.582906    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:27.592230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:27.583379    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:55:27.597730 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:27.596509    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:55:28.301563 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:28.301502    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": context deadline exceeded"
Oct 30 15:55:28.426696 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:28.426665    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:55:28.448219 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:28.435681    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:55:28.448219 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:28.435747    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:55:44.421525 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:44.421469    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:44.442089 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:44.440448    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:44.442089 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:44.440895    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:55:44.477481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:44.477426    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:44.478055 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:44.478025    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:55:44.478934 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:44.478901    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:55:44.620372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:44.601470    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:44.885428 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:44.884482    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:45.183497 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.183447    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:45.431682 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.431638    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:45.433870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.433805    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:55:45.443466 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.443427    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:45.444045 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.443675    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:45.444914 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.444629    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:55:45.445528 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.445511    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:55:45.528467 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.521451    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:55:45.551885 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.527435    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:55:45.576806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.528757    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:45.577447 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.552169    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:55:45.577685 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.577666    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:55:45.578260 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.578243    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:55:45.612515 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.612465    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:55:45.613276 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.613251    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:55:45.801428 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.732899    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:45.887855 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:45.887810    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:46.048917 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:46.048880    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:55:46.103479 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:46.078567    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:55:46.103479 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:46.078630    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:46.458686 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:46.458627    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:55:46.466597 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:46.466554    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:55:46.531316 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:46.458573    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:55:46.577064 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:46.576963    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:46.577530 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:46.577318    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:46.578975 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:46.577384    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:55:46.620513 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:46.620469    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:55:47.056473 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:47.056426    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:47.524738 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:47.489272    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:47.524738 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:47.490370    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:55:47.579518 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:47.534823    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:47.590371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:47.580456    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:47.608563 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:47.608520    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:47.609135 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:47.609109    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:55:47.680942 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:47.677507    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:55:48.112370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:48.088846    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:55:48.113801 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:48.109764    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:48.184364 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:48.172898    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:55:48.184364 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:48.174838    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:55:48.184364 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:48.177417    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:55:48.184364 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:48.177947    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:55:48.193367 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:48.184933    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:55:48.229684 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:48.229508    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:55:48.379474 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:48.379397    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:55:48.388550 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:55:48.381660    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:55:48.401224 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:55:48.394792    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:56:00.541371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:00.540452    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:00.615408 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:00.614456    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:00.619461 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:00.619425    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:56:00.683498 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:00.683450    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:01.543387 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:01.541455    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:01.543387 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:01.542384    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:56:01.620497 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:01.620456    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:56:01.621072 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:01.621049    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:56:02.040373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:02.039684    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:02.282375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:02.227690    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/health\": context deadline exceeded"
Oct 30 15:56:02.341429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:02.307453    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:56:02.415376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:02.326674    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:56:02.608430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:02.355137    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:56:02.608430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:02.506417    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:02.608430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:02.544489    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:02.654373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:02.653528    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:56:02.679373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:02.654884    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:56:02.874103 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:02.874003    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:03.412898 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.412856    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:56:03.434370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.432970    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:56:03.625530 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.625481    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:56:03.664076 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.626090    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:03.704379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.626125    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:56:03.742930 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.626160    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:03.743378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.704677    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:03.743847 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.704716    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:56:03.744271 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.716790    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:56:03.744545 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.744523    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:56:03.744690 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.744678    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:56:03.744826 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.744816    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:56:03.849631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.849597    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:56:03.849893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.849864    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:03.885032 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.884992    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:03.885685 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:03.885662    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:56:04.359190 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:04.359161    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:56:04.420377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:04.420277    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:56:04.510405 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:04.509236    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:56:04.557370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:04.552721    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:56:04.643256 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:04.627101    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:56:04.643977 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:04.643945    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:56:04.707301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:04.707159    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:04.764653 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:04.764622    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:56:04.764653 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:04.764688    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:56:15.633649 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:15.632471    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:15.695289 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:15.693473    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:15.703546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:15.703472    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:15.772490 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:15.772439    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:15.828378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:15.796246    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:16.389373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:16.388429    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:16.428377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:16.424737    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:16.636382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:16.634497    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:16.636382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:16.635541    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:56:16.694549 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:16.694493    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:56:16.813373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:16.777599    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:17.016446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:16.992489    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:56:17.031380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:17.030798    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:56:17.031380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:17.031144    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:56:17.031380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:17.031185    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:56:17.488373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:17.486442    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:56:17.507995 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:17.507956    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:56:17.646748 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:17.646701    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:17.651447 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:17.651411    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:56:17.812456 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:17.809518    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:56:17.820401 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:17.816646    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:56:18.054785 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.054743    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:56:18.093773 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.093672    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": context deadline exceeded"
Oct 30 15:56:18.422430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.335722    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:56:18.585822 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.571490    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:56:18.679434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.650572    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:18.679434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.659518    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:56:18.682716 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.681566    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:56:18.690101 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.690071    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:56:18.785310 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.785270    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:56:18.785635 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.785601    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:18.788324 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.788287    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:18.928228 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.888982    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:56:18.945806 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:56:18.945684    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:56:18.999367 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.999318    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:56:18.999561 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:18.999418    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:19.073511 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.073462    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:19.512984 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.512946    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:56:19.514647 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.514613    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:56:19.515507 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.515469    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:56:19.615529 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.615481    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:56:19.675374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.671939    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:19.675374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.672227    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:56:19.695496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.695450    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:19.696520 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.696493    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:56:19.818669 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.808598    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:19.938369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:19.935444    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:20.195542 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:20.195503    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:56:20.207677 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:20.207647    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:56:20.221688 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:20.221494    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:56:20.223070 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:20.223045    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:56:20.223509 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:20.223490    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:56:20.230278 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:20.230256    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:56:20.243008 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:20.242980    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:56:20.253213 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:20.252869    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:56:20.256667 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:20.253780    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:56:22.496261 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.496224    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-89g6q-gkgrp"]
Oct 30 15:56:22.500628 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.500605    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.519322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.519286    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-89g6q-gkgrp"]
Oct 30 15:56:22.527293 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.527264    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 15:56:22.551180 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.551153    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mhkxg\""
Oct 30 15:56:22.552382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.552343    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 15:56:22.620804 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.620766    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-trusted-ca-bundle\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.620980 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.620830    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-service-ca-bundle\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.620980 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.620932    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jftlz\" (UniqueName: \"kubernetes.io/projected/8fdee16a-c007-40f8-9337-c896bc2c644e-kube-api-access-jftlz\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.620980 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.620968    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8fdee16a-c007-40f8-9337-c896bc2c644e-archives-path\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.722654 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.722618    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-service-ca-bundle\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.722847 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.722715    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jftlz\" (UniqueName: \"kubernetes.io/projected/8fdee16a-c007-40f8-9337-c896bc2c644e-kube-api-access-jftlz\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.722847 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.722751    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8fdee16a-c007-40f8-9337-c896bc2c644e-archives-path\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.722993 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.722974    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-trusted-ca-bundle\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.723183 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.723160    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8fdee16a-c007-40f8-9337-c896bc2c644e-archives-path\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.723446 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.723422    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-service-ca-bundle\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.724054 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.724035    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-trusted-ca-bundle\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.746044 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.746015    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jftlz\" (UniqueName: \"kubernetes.io/projected/8fdee16a-c007-40f8-9337-c896bc2c644e-kube-api-access-jftlz\") pod \"periodic-gathering-89g6q-gkgrp\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") " pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:22.814121 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:22.814031    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:56:23.116474 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:56:23.116444    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fdee16a_c007_40f8_9337_c896bc2c644e.slice/crio-d0a529fe930e7935de0f6201f07d9ea4c804d64abbc93c3851565b90f0404fa0 WatchSource:0}: Error finding container d0a529fe930e7935de0f6201f07d9ea4c804d64abbc93c3851565b90f0404fa0: Status 404 returned error can't find the container with id d0a529fe930e7935de0f6201f07d9ea4c804d64abbc93c3851565b90f0404fa0
Oct 30 15:56:23.117340 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:23.117288    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-89g6q-gkgrp"]
Oct 30 15:56:23.917970 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:23.917929    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-89g6q-gkgrp" event={"ID":"8fdee16a-c007-40f8-9337-c896bc2c644e","Type":"ContainerStarted","Data":"590e4779660784159d2f6f0e1e4d92885eb95667e3316d9d02728dab257a0b63"}
Oct 30 15:56:23.918530 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:23.917994    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-89g6q-gkgrp" event={"ID":"8fdee16a-c007-40f8-9337-c896bc2c644e","Type":"ContainerStarted","Data":"d0a529fe930e7935de0f6201f07d9ea4c804d64abbc93c3851565b90f0404fa0"}
Oct 30 15:56:23.959543 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:23.959490    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-89g6q-gkgrp" podStartSLOduration=1.959476581 podStartE2EDuration="1.959476581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:56:23.957318158 +0000 UTC m=+1093.392922508" watchObservedRunningTime="2026-10-30 15:56:23.959476581 +0000 UTC m=+1093.395080929"
Oct 30 15:56:26.155213 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.155172    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"]
Oct 30 15:56:26.160167 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.160143    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.177780 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.177749    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"]
Oct 30 15:56:26.184602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.184562    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 15:56:26.184602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.184593    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 15:56:26.296525 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.296492    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cb15f4b0-p5th6\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.296712 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.296624    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4kwt\" (UniqueName: \"kubernetes.io/projected/86e0f877-9d9f-4716-8e8a-707d2096b466-kube-api-access-l4kwt\") pod \"maas-api-revoke-sub-cb15f4b0-p5th6\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.296831 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.296817    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cb15f4b0-p5th6\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.398160 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.398124    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cb15f4b0-p5th6\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.398400 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.398224    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4kwt\" (UniqueName: \"kubernetes.io/projected/86e0f877-9d9f-4716-8e8a-707d2096b466-kube-api-access-l4kwt\") pod \"maas-api-revoke-sub-cb15f4b0-p5th6\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.398479 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.398409    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cb15f4b0-p5th6\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.398905 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.398876    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cb15f4b0-p5th6\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.401084 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.401059    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cb15f4b0-p5th6\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.422534 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.422440    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4kwt\" (UniqueName: \"kubernetes.io/projected/86e0f877-9d9f-4716-8e8a-707d2096b466-kube-api-access-l4kwt\") pod \"maas-api-revoke-sub-cb15f4b0-p5th6\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.475455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.475410    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:26.798664 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.798636    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"]
Oct 30 15:56:26.944238 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:26.944198    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6" event={"ID":"86e0f877-9d9f-4716-8e8a-707d2096b466","Type":"ContainerStarted","Data":"4a0037da082e819df23bef20ea727dc367631b43993e56d52d2557ea6d400e49"}
Oct 30 15:56:27.953457 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:27.953419    4585 generic.go:356] "Generic (PLEG): container finished" podID="86e0f877-9d9f-4716-8e8a-707d2096b466" containerID="c033b8c410842cf42a30294c4c996ec9674cb0a65e22b2d36963d8e3bee57a80" exitCode=0
Oct 30 15:56:27.953841 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:27.953501    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6" event={"ID":"86e0f877-9d9f-4716-8e8a-707d2096b466","Type":"ContainerDied","Data":"c033b8c410842cf42a30294c4c996ec9674cb0a65e22b2d36963d8e3bee57a80"}
Oct 30 15:56:29.121236 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.121210    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:29.243049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.242950    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86e0f877-9d9f-4716-8e8a-707d2096b466-kube-api-access-l4kwt\" (UniqueName: \"kubernetes.io/projected/86e0f877-9d9f-4716-8e8a-707d2096b466-kube-api-access-l4kwt\") pod \"86e0f877-9d9f-4716-8e8a-707d2096b466\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") "
Oct 30 15:56:29.243219 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.243169    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-xks-ca\") pod \"86e0f877-9d9f-4716-8e8a-707d2096b466\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") "
Oct 30 15:56:29.243411 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.243389    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-openshift-ca\") pod \"86e0f877-9d9f-4716-8e8a-707d2096b466\" (UID: \"86e0f877-9d9f-4716-8e8a-707d2096b466\") "
Oct 30 15:56:29.243866 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.243840    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-openshift-ca" pod "86e0f877-9d9f-4716-8e8a-707d2096b466" (UID: "86e0f877-9d9f-4716-8e8a-707d2096b466"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:56:29.244088 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.244071    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:56:29.246286 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.246260    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86e0f877-9d9f-4716-8e8a-707d2096b466-kube-api-access-l4kwt" pod "86e0f877-9d9f-4716-8e8a-707d2096b466" (UID: "86e0f877-9d9f-4716-8e8a-707d2096b466"). InnerVolumeSpecName "kube-api-access-l4kwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:56:29.249527 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.249501    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-xks-ca" pod "86e0f877-9d9f-4716-8e8a-707d2096b466" (UID: "86e0f877-9d9f-4716-8e8a-707d2096b466"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:56:29.345420 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.345387    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4kwt\" (UniqueName: \"kubernetes.io/projected/86e0f877-9d9f-4716-8e8a-707d2096b466-kube-api-access-l4kwt\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:56:29.345420 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.345413    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/86e0f877-9d9f-4716-8e8a-707d2096b466-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:56:29.968526 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.968492    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"
Oct 30 15:56:29.968526 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.968513    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6" event={"ID":"86e0f877-9d9f-4716-8e8a-707d2096b466","Type":"ContainerDied","Data":"4a0037da082e819df23bef20ea727dc367631b43993e56d52d2557ea6d400e49"}
Oct 30 15:56:29.968727 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:29.968580    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a0037da082e819df23bef20ea727dc367631b43993e56d52d2557ea6d400e49"
Oct 30 15:56:35.205414 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:35.197696    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:35.263372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:35.243829    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:35.382372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:35.373213    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:56:35.563152 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:35.563024    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:35.606522 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:35.606472    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:35.688088 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:35.688025    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:35.700453 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:35.700414    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/health\": context deadline exceeded"
Oct 30 15:56:35.936506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:35.936454    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:36.101407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.092457    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:36.268606 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.266471    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:36.268606 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.267275    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:56:36.375933 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.375890    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:56:36.377023 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.376995    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:56:36.642437 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.607966    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:56:36.642437 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.608761    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:56:36.762380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.752579    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:56:36.881588 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.847135    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:56:36.943379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.940458    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:36.943379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:36.941225    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:56:37.077969 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.077937    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:56:37.078275 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.078227    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:37.130379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.119504    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:56:37.130379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.119584    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:37.132262 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.130681    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:37.132262 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.131124    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:56:37.274951 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.274855    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:37.290390 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.276110    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:37.406371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.400215    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:37.548001 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.547887    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:37.615546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.615494    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:56:37.904432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.851148    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:37.928192 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.909623    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"]
Oct 30 15:56:37.928192 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.914389    4585 cpu_manager.go:414] "containerMap: removing container" podUID="86e0f877-9d9f-4716-8e8a-707d2096b466" containerName="revoke-keys"
Oct 30 15:56:37.928192 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.914419    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86e0f877-9d9f-4716-8e8a-707d2096b466" containerName="revoke-keys"
Oct 30 15:56:37.952504 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.952447    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:37.972331 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.972286    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:37.980714 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:37.980682    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="86e0f877-9d9f-4716-8e8a-707d2096b466" containerName="revoke-keys"
Oct 30 15:56:38.151373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.138608    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:38.216489 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.215151    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"]
Oct 30 15:56:38.220645 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.220558    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.305949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.275727    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:38.305949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.292612    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:56:38.305949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.294172    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:56:38.305949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.296890    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:38.305949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.297184    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-ba9db9\"/\"distinct-w5-ba9db9-kserve-self-signed-certs\""
Oct 30 15:56:38.368422 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.360573    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-dshm\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.466181 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.465937    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tls-certs\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.466181 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.466035    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-home\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.466181 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.466078    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tmp-dir\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.466588 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.466274    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-model-cache\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.466588 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.466322    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dpqs\" (UniqueName: \"kubernetes.io/projected/c1a469f1-e1a1-4800-8827-14e7ce0501cc-kube-api-access-6dpqs\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.466711 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.361443    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:56:38.466711 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.466680    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:38.518496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.518441    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:56:38.570399 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.564765    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:38.570399 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.565531    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:56:38.574370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.571406    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-dshm\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.574370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.571511    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tls-certs\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.574370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.571575    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-home\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.574370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.571617    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tmp-dir\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.574370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.571760    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-model-cache\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.574370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.571797    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dpqs\" (UniqueName: \"kubernetes.io/projected/c1a469f1-e1a1-4800-8827-14e7ce0501cc-kube-api-access-6dpqs\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.667395 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.666958    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-home\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.707386 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.697226    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tmp-dir\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.707386 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.697544    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-model-cache\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:38.710783 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.710744    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:56:38.912237 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.912193    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:38.932371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.930898    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dpqs\" (UniqueName: \"kubernetes.io/projected/c1a469f1-e1a1-4800-8827-14e7ce0501cc-kube-api-access-6dpqs\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:39.023382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:38.973083    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tls-certs\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:39.045586 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.045541    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:56:39.071582 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.065792    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-dshm\") pod \"distinct-w5-ba9db9-kserve-66556578b5-bgv9d\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") " pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:39.113800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.113758    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:39.168741 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.152321    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:56:39.317794 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.315596    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:39.369120 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.369072    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:56:39.405924 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.405885    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:56:39.406211 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.406192    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:56:39.444320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.443294    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:56:39.445379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.445322    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:56:39.450534 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.450515    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:56:39.461119 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.455653    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:56:39.461119 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.456249    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:56:39.463516 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.462698    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:56:39.725933 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.725900    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"]
Oct 30 15:56:39.726304 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:56:39.726277    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1a469f1_e1a1_4800_8827_14e7ce0501cc.slice/crio-063b40b252660a64f23c251cc1d9ce5ff173496a52fea10290b0277e76be7b03 WatchSource:0}: Error finding container 063b40b252660a64f23c251cc1d9ce5ff173496a52fea10290b0277e76be7b03: Status 404 returned error can't find the container with id 063b40b252660a64f23c251cc1d9ce5ff173496a52fea10290b0277e76be7b03
Oct 30 15:56:39.868135 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.867546    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" event={"ID":"c1a469f1-e1a1-4800-8827-14e7ce0501cc","Type":"ContainerStarted","Data":"f0c4621445ec8e12b12a9a24d7622193a17d8dc075ad19ede1439a6cf7646c03"}
Oct 30 15:56:39.868135 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.867593    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" event={"ID":"c1a469f1-e1a1-4800-8827-14e7ce0501cc","Type":"ContainerStarted","Data":"063b40b252660a64f23c251cc1d9ce5ff173496a52fea10290b0277e76be7b03"}
Oct 30 15:56:39.911061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:39.911008    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podStartSLOduration=2.91099465 podStartE2EDuration="2.91099465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:56:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:56:39.908187087 +0000 UTC m=+1109.343791434" watchObservedRunningTime="2026-10-30 15:56:39.91099465 +0000 UTC m=+1109.346599013"
Oct 30 15:56:40.118757 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:40.118660    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:47.961362 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:56:47.961311    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:56:49.117913 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:49.117873    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:49.131884 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:49.131857    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:49.953938 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:49.953903    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:57.833650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:57.827590    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:56:57.918370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:57.917449    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:57.956374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:57.950640    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:58.161376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:58.160594    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:58.298003 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:58.297947    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:58.382244 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:58.380453    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:58.469595 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:58.469483    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:58.544091 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:58.544045    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:58.839432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:58.833470    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:58.839432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:58.834297    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:56:58.918487 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:58.918443    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:58.919258 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:58.919233    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:56:59.070374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.041580    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:59.070374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.042105    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:56:59.106466 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.079445    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:56:59.164585 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.164545    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:56:59.165383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.165337    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:56:59.411464 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.396460    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:59.412565 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.412532    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:56:59.440647 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.413929    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:56:59.484047 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.483992    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:59.484528 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.484507    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:56:59.531497 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.531450    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:59.546500 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.546448    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:56:59.557383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:56:59.547141    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:57:00.143373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.114692    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:57:00.181383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.177649    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:00.181383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.178815    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:57:00.182113 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.182071    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:57:00.191398 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.187403    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/health\": context deadline exceeded"
Oct 30 15:57:00.208376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.195601    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:57:00.310555 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.310517    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:00.527014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.401798    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:57:00.732517 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.467412    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:57:00.732835 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.512981    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:57:01.142336 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:00.912453    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:01.201580 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.201531    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:01.224559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.224511    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:01.262541 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.262496    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:57:01.262820 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.262787    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:01.301654 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.301606    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:01.331828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.331783    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:01.332139 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.332113    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:57:01.537532 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.537479    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:57:01.538207 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.538171    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:01.578939 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.578904    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:57:01.796268 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.796166    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:01.925432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:01.921510    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:02.560296 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:02.557706    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:57:02.582374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:02.582087    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:57:02.587388 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:02.584395    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:02.587388 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:02.584814    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:57:02.587388 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:02.585107    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:02.587388 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:02.586497    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:02.587388 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:02.586772    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/health\": context deadline exceeded"
Oct 30 15:57:02.587388 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:02.586823    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:57:02.587388 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:02.587056    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:57:02.844737 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:02.841755    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:03.566381 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.565449    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:03.593106 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.593056    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:57:03.594617 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.594577    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:03.595856 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.595832    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:57:03.597053 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.597030    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:57:03.613756 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.613712    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:57:03.615329 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.615294    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:57:03.639995 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.616602    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:57:03.741504 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.741465    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:57:03.741821 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.741787    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:03.848153 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:03.848041    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:04.415553 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:04.415515    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:57:04.419236 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:04.419213    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:57:04.421827 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:04.421796    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:57:04.430376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:04.428910    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:57:04.506573 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:04.505823    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:57:04.507426 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:04.506795    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:57:04.507740 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:04.507723    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:57:04.546178 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:04.546148    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:57:04.550410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:04.547567    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:57:12.448717 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.448675    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"]
Oct 30 15:57:12.453394 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.453370    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.484907 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.484870    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"]
Oct 30 15:57:12.627725 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.627685    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.627911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.627819    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.627911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.627850    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.627911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.627874    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.628014 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.627907    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsqr9\" (UniqueName: \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-kube-api-access-lsqr9\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.628063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.628042    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e084541-731b-46dc-a2e0-dd989b1b36a6-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.628125 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.628113    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.628185 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.628170    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.628269 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.628257    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729218 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729115    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e084541-731b-46dc-a2e0-dd989b1b36a6-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729448 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729217    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729448 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729257    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729448 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729388    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729626 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729523    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729626 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729616    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729735 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729619    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729735 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729668    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729735 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729693    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729735 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729722    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsqr9\" (UniqueName: \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-kube-api-access-lsqr9\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729760    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.729947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729879    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.730054 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.729984    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e084541-731b-46dc-a2e0-dd989b1b36a6-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.730092 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.730073    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.731866 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.731844    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.732215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.732197    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.756631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.756604    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.756770 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.756743    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsqr9\" (UniqueName: \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-kube-api-access-lsqr9\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:12.767588 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:12.767562    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:13.089333 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:57:13.089301    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e084541_731b_46dc_a2e0_dd989b1b36a6.slice/crio-6b0bb9518a50b5efd3ba0baf93499faa9026808e1a9abacac4caff4efa30dd9a WatchSource:0}: Error finding container 6b0bb9518a50b5efd3ba0baf93499faa9026808e1a9abacac4caff4efa30dd9a: Status 404 returned error can't find the container with id 6b0bb9518a50b5efd3ba0baf93499faa9026808e1a9abacac4caff4efa30dd9a
Oct 30 15:57:13.090545 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:13.090518    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"]
Oct 30 15:57:13.092488 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:13.092454    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:57:13.092591 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:13.092529    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:57:13.092591 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:13.092560    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:57:13.600548 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:13.600514    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" event={"ID":"8e084541-731b-46dc-a2e0-dd989b1b36a6","Type":"ContainerStarted","Data":"888d9afa53ca835e0e432cb1662b5087e01fd6be46c3c4e07d9b58b6701ad720"}
Oct 30 15:57:13.600952 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:13.600569    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" event={"ID":"8e084541-731b-46dc-a2e0-dd989b1b36a6","Type":"ContainerStarted","Data":"6b0bb9518a50b5efd3ba0baf93499faa9026808e1a9abacac4caff4efa30dd9a"}
Oct 30 15:57:13.767756 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:13.767723    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:14.770770 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:14.770732    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 15:57:14.771246 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:14.770793    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 15:57:15.768406 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:15.768369    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 15:57:15.768594 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:15.768425    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 15:57:16.612671 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.612606    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" podStartSLOduration=4.612585178 podStartE2EDuration="4.612585178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:57:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:57:13.65910247 +0000 UTC m=+1143.094706818" watchObservedRunningTime="2026-10-30 15:57:16.612585178 +0000 UTC m=+1146.048189533"
Oct 30 15:57:16.613088 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.612824    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"]
Oct 30 15:57:16.617304 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.617281    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.647373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.641615    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 15:57:16.647373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.641783    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bvrzq\""
Oct 30 15:57:16.647373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.641854    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 15:57:16.647373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.641917    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 15:57:16.681224 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.681186    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"]
Oct 30 15:57:16.708445 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.708403    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.708633 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.708616    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnrts\" (UniqueName: \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-kube-api-access-pnrts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.708746 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.708734    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.709037 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.709010    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.709086 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.709056    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.709215 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.709192    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/311f2664-3b46-4ff7-bec0-b553d0204153-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.709275 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.709258    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.768956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.768922    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 15:57:16.769144 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.768979    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 15:57:16.811232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.810675    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.811232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.810913    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.811232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.810941    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnrts\" (UniqueName: \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-kube-api-access-pnrts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.811232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.810985    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.811232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.811188    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.811232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.811226    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.811733 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.811290    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/311f2664-3b46-4ff7-bec0-b553d0204153-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.813076 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.813038    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.814004 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.813978    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/311f2664-3b46-4ff7-bec0-b553d0204153-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.814970 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.814949    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.815887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.815846    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.818664 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.816321    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.854158 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.854067    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnrts\" (UniqueName: \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-kube-api-access-pnrts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.854933 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.854910    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4ks2f\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:16.971373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:16.970447    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:17.385970 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:17.385943    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"]
Oct 30 15:57:17.386703 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:17.386052    4585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 15:57:17.638429 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:17.634905    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f" event={"ID":"311f2664-3b46-4ff7-bec0-b553d0204153","Type":"ContainerStarted","Data":"646aa2924b60f0d5d705916981d1af45a3a188848b8b251924d84794e099e030"}
Oct 30 15:57:17.772371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:17.771212    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 15:57:17.772371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:17.771272    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 15:57:17.994774 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:57:17.994740    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:57:18.769367 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:18.768929    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 15:57:18.769367 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:18.768992    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 15:57:20.031908 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:20.030502    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:20.031908 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:20.031760    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:20.033004 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:20.032983    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:57:20.770849 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:20.770808    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:57:20.770990 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:20.770874    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:57:21.699683 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:21.699647    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f" event={"ID":"311f2664-3b46-4ff7-bec0-b553d0204153","Type":"ContainerStarted","Data":"45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab"}
Oct 30 15:57:21.700165 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:21.699949    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:21.702223 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:21.702193    4585 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-4ks2f container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 15:57:21.702336 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:21.702260    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f" podUID="311f2664-3b46-4ff7-bec0-b553d0204153" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 15:57:21.771656 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:21.771597    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f" podStartSLOduration=2.388497074 podStartE2EDuration="5.771580545s" totalImagesPullingTime="3.383083471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:57:21.756028925 +0000 UTC m=+1151.191633291" watchObservedRunningTime="2026-10-30 15:57:21.771580545 +0000 UTC m=+1151.207184893"
Oct 30 15:57:22.706427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:22.706398    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 15:57:24.119373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:24.118890    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:24.129371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:24.126698    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:24.519079 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:24.517977    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:24.551764 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:24.550745    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:25.126544 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:25.126480    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:25.128031 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:25.128005    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:57:25.130208 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:25.130182    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:25.130753 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:25.130735    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:57:25.211796 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:25.201301    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:25.212464 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:25.212427    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:57:25.273373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:25.262860    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:57:25.592563 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:25.569540    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:25.752374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:25.648189    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:25.984487 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:25.963612    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:26.052369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.048775    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:26.052369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.050448    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:57:26.101373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.076570    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:26.113499 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.089067    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:57:26.242432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.126699    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:57:26.242432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.171427    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:57:26.242432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.171674    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:57:26.242432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.236546    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:26.242432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.239228    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:57:26.327775 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.327725    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:26.357087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.357054    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:57:26.723913 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.723872    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 15:57:26.725955 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.725928    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:57:26.943374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.939738    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:57:27.010435 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:26.998043    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:27.111538 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:27.111485    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:27.173375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:27.172461    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:27.173589 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:27.173422    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:57:27.193243 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:27.169462    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:57:27.243597 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:27.243526    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:57:27.369870 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:27.368426    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:57:27.622139 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:27.621391    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:57:27.637485 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:27.633610    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:57:27.737661 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:27.737606    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:27.957953 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:27.957914    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:28.051377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:28.026015    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:28.152499 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:28.152450    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:57:28.403435 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:28.350866    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": context deadline exceeded"
Oct 30 15:57:28.405866 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:28.404591    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:57:28.419380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:28.404742    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:57:28.420839 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:28.404850    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:57:28.468397 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:28.405468    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:57:28.560833 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:28.507152    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:28.668515 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:28.656383    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:28.803371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:28.761711    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:29.075433 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:29.074236    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:29.128379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:29.101605    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": context deadline exceeded"
Oct 30 15:57:29.384434 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:29.328940    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:29.528728 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:29.528673    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:57:29.560415 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:29.529300    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:57:29.563932 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:29.528615    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:29.585267 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:29.564159    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:29.591978 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:29.572400    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/health\": context deadline exceeded"
Oct 30 15:57:29.592265 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:29.572452    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:57:29.875614 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:29.874487    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:30.095774 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.095721    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:30.162660 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.131305    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/health\": context deadline exceeded"
Oct 30 15:57:30.250380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.239079    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:30.441232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.441117    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:57:30.553372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.552449    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:30.582368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.579456    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 15:57:30.597526 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.588868    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:30.597526 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.589276    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:57:30.597872 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.597834    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:30.888937 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.888251    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:30.952563 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.952070    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:57:30.952563 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:30.952132    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:31.171505 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.170511    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:31.249506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.248587    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:57:31.288880 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.263240    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:31.459968 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.441623    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:31.561600 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.561559    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 15:57:31.632201 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.592510    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:31.632201 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.594283    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:31.632201 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.595048    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:57:31.705876 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.704313    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:31.718736 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.717464    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:57:31.728019 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.726791    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:57:31.787846 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.782719    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:57:31.832435 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.831180    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:57:31.841971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.841942    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:57:31.852374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.852320    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:57:31.900379 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.896973    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:57:31.923922 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.923892    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:57:31.999875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:31.999791    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:57:33.262132 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:33.262097    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"]
Oct 30 15:57:33.292165 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:33.292133    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"]
Oct 30 15:57:36.281630 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:36.281595    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"]
Oct 30 15:57:41.758958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:41.758919    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"]
Oct 30 15:57:45.843954 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:45.843919    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"]
Oct 30 15:57:46.760776 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:46.760708    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb" podUID="d18ea2be-fa5a-45c8-8589-e081450576d4" containerName="payload-pre-processing" containerID="cri-o://945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0" gracePeriod=25
Oct 30 15:57:47.966515 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:57:47.966468    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:57:48.288671 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.288568    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" containerID="cri-o://0593593208e1643e0a0a94cf0093456b9862fec39064160d258338d867d6edd9" gracePeriod=45
Oct 30 15:57:48.313622 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.313590    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main" containerID="cri-o://cbfd39155c83bfe56b02f929853592cc7947679e828b63c4290177665148b272" gracePeriod=45
Oct 30 15:57:48.596082 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.595883    4585 generic.go:356] "Generic (PLEG): container finished" podID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerID="cbfd39155c83bfe56b02f929853592cc7947679e828b63c4290177665148b272" exitCode=0
Oct 30 15:57:48.596082 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.595971    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" event={"ID":"c0ae31d1-0e25-441b-8257-4871a18ed6ee","Type":"ContainerDied","Data":"cbfd39155c83bfe56b02f929853592cc7947679e828b63c4290177665148b272"}
Oct 30 15:57:48.598950 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.598846    4585 generic.go:356] "Generic (PLEG): container finished" podID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerID="0593593208e1643e0a0a94cf0093456b9862fec39064160d258338d867d6edd9" exitCode=0
Oct 30 15:57:48.599297 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.598926    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" event={"ID":"dfa4b03a-3d65-4fdf-99cc-704df480ca67","Type":"ContainerDied","Data":"0593593208e1643e0a0a94cf0093456b9862fec39064160d258338d867d6edd9"}
Oct 30 15:57:48.645769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.645743    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:57:48.650044 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.650017    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:57:48.805886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.805850    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tmp-dir\") pod \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") "
Oct 30 15:57:48.806093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.805923    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-model-cache\") pod \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") "
Oct 30 15:57:48.806093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.805960    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tls-certs\") pod \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") "
Oct 30 15:57:48.806093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.805989    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tls-certs\") pod \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") "
Oct 30 15:57:48.806093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.806084    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0ae31d1-0e25-441b-8257-4871a18ed6ee-kube-api-access-zdsrh\" (UniqueName: \"kubernetes.io/projected/c0ae31d1-0e25-441b-8257-4871a18ed6ee-kube-api-access-zdsrh\") pod \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") "
Oct 30 15:57:48.806308 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.806157    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tmp-dir" pod "c0ae31d1-0e25-441b-8257-4871a18ed6ee" (UID: "c0ae31d1-0e25-441b-8257-4871a18ed6ee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:48.806584 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.806558    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-model-cache" pod "c0ae31d1-0e25-441b-8257-4871a18ed6ee" (UID: "c0ae31d1-0e25-441b-8257-4871a18ed6ee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:48.806676 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.806652    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-home\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-home\") pod \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") "
Oct 30 15:57:48.806728 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.806690    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tmp-dir\") pod \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") "
Oct 30 15:57:48.806917 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.806893    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfa4b03a-3d65-4fdf-99cc-704df480ca67-kube-api-access-gvzjn\" (UniqueName: \"kubernetes.io/projected/dfa4b03a-3d65-4fdf-99cc-704df480ca67-kube-api-access-gvzjn\") pod \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") "
Oct 30 15:57:48.806998 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.806905    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-home" pod "dfa4b03a-3d65-4fdf-99cc-704df480ca67" (UID: "dfa4b03a-3d65-4fdf-99cc-704df480ca67"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:48.807054 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.807031    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-model-cache\") pod \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") "
Oct 30 15:57:48.807117 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.807071    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-dshm\") pod \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\" (UID: \"dfa4b03a-3d65-4fdf-99cc-704df480ca67\") "
Oct 30 15:57:48.807117 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.807110    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-home\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-home\") pod \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") "
Oct 30 15:57:48.807255 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.807243    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-dshm\") pod \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\" (UID: \"c0ae31d1-0e25-441b-8257-4871a18ed6ee\") "
Oct 30 15:57:48.807646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.807289    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-model-cache" pod "dfa4b03a-3d65-4fdf-99cc-704df480ca67" (UID: "dfa4b03a-3d65-4fdf-99cc-704df480ca67"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:48.807646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.807607    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tmp-dir" pod "dfa4b03a-3d65-4fdf-99cc-704df480ca67" (UID: "dfa4b03a-3d65-4fdf-99cc-704df480ca67"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:48.807905 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.807881    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-home" pod "c0ae31d1-0e25-441b-8257-4871a18ed6ee" (UID: "c0ae31d1-0e25-441b-8257-4871a18ed6ee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:48.809669 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.809647    4585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-home\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.809782 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.809673    4585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tmp-dir\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.809782 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.809689    4585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-model-cache\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.809782 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.809702    4585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-home\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.809782 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.809713    4585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tmp-dir\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.809782 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.809727    4585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-model-cache\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.809782 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.809726    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tls-certs" pod "c0ae31d1-0e25-441b-8257-4871a18ed6ee" (UID: "c0ae31d1-0e25-441b-8257-4871a18ed6ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:57:48.810105 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.809946    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ae31d1-0e25-441b-8257-4871a18ed6ee-kube-api-access-zdsrh" pod "c0ae31d1-0e25-441b-8257-4871a18ed6ee" (UID: "c0ae31d1-0e25-441b-8257-4871a18ed6ee"). InnerVolumeSpecName "kube-api-access-zdsrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:57:48.810205 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.810183    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tls-certs" pod "dfa4b03a-3d65-4fdf-99cc-704df480ca67" (UID: "dfa4b03a-3d65-4fdf-99cc-704df480ca67"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:57:48.810781 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.810761    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-dshm" pod "c0ae31d1-0e25-441b-8257-4871a18ed6ee" (UID: "c0ae31d1-0e25-441b-8257-4871a18ed6ee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:48.810886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.810757    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-dshm" pod "dfa4b03a-3d65-4fdf-99cc-704df480ca67" (UID: "dfa4b03a-3d65-4fdf-99cc-704df480ca67"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:48.811284 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.811261    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa4b03a-3d65-4fdf-99cc-704df480ca67-kube-api-access-gvzjn" pod "dfa4b03a-3d65-4fdf-99cc-704df480ca67" (UID: "dfa4b03a-3d65-4fdf-99cc-704df480ca67"). InnerVolumeSpecName "kube-api-access-gvzjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:57:48.910841 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.910809    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvzjn\" (UniqueName: \"kubernetes.io/projected/dfa4b03a-3d65-4fdf-99cc-704df480ca67-kube-api-access-gvzjn\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.910841 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.910840    4585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa4b03a-3d65-4fdf-99cc-704df480ca67-dshm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.911061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.910854    4585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0ae31d1-0e25-441b-8257-4871a18ed6ee-dshm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.911061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.910866    4585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa4b03a-3d65-4fdf-99cc-704df480ca67-tls-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.911061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.910880    4585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ae31d1-0e25-441b-8257-4871a18ed6ee-tls-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:48.911061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:48.910893    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdsrh\" (UniqueName: \"kubernetes.io/projected/c0ae31d1-0e25-441b-8257-4871a18ed6ee-kube-api-access-zdsrh\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:49.566219 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.566171    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 15:57:49.607410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.607380    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"
Oct 30 15:57:49.607410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.607390    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm" event={"ID":"c0ae31d1-0e25-441b-8257-4871a18ed6ee","Type":"ContainerDied","Data":"046f75a42df38c3e7006e037e67873e32671f79010604a38efc3cc8f7858d620"}
Oct 30 15:57:49.607632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.607439    4585 scope.go:122] "RemoveContainer" containerID="cbfd39155c83bfe56b02f929853592cc7947679e828b63c4290177665148b272"
Oct 30 15:57:49.608892 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.608873    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"
Oct 30 15:57:49.608970 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.608887    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9" event={"ID":"dfa4b03a-3d65-4fdf-99cc-704df480ca67","Type":"ContainerDied","Data":"215714ec4db993ef1981dbae6d3178ae4a5a1c062a0f438697cb961b33cb9e94"}
Oct 30 15:57:49.620016 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.619800    4585 scope.go:122] "RemoveContainer" containerID="0593593208e1643e0a0a94cf0093456b9862fec39064160d258338d867d6edd9"
Oct 30 15:57:49.660536 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.660503    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"]
Oct 30 15:57:49.672470 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.672442    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/simulator-w0-b63c3c-kserve-5998f5d59-562jm"]
Oct 30 15:57:49.700875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.700842    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"]
Oct 30 15:57:49.704878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:49.704846    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-b63c3c/distinct-2-w0-b63c3c-kserve-7fc78fb5f4-rqrv9"]
Oct 30 15:57:50.742373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:50.741872    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" path="/var/lib/kubelet/pods/c0ae31d1-0e25-441b-8257-4871a18ed6ee/volumes"
Oct 30 15:57:50.742926 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:50.742644    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" path="/var/lib/kubelet/pods/dfa4b03a-3d65-4fdf-99cc-704df480ca67/volumes"
Oct 30 15:57:51.332372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:51.329005    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" containerID="cri-o://f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116" gracePeriod=45
Oct 30 15:57:51.803899 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:51.792932    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:57:51.803899 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:51.793006    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:52.060437 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.056503    4585 patch_prober.go:29] interesting pod/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:57:52.094114 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.093918    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:52.110280 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.094901    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:52.136281 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.135539    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:57:52.136605 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.136558    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:52.171692 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.171317    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": EOF"
Oct 30 15:57:52.177586 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.177445    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:57:52.177586 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.177512    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:52.309012 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.193464    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:57:52.419756 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.335537    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": dial tcp 10.134.0.64:8000: connect: connection refused"
Oct 30 15:57:52.419756 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.335684    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:57:52.566495 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.566444    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:52.705681 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.705584    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"]
Oct 30 15:57:52.894433 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:52.865850    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:53.129194 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.129073    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:53.146254 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.144027    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:57:53.147222 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.147197    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:57:53.172800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.172741    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": dial tcp 10.134.0.64:8000: connect: connection refused"
Oct 30 15:57:53.307433 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.298156    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:57:53.731025 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.730995    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:57:53.753761 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.753725    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-home\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-home\") pod \"4067e045-ff27-4da7-99e3-e6e6026b182e\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") "
Oct 30 15:57:53.753944 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.753865    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-tmp-dir\") pod \"4067e045-ff27-4da7-99e3-e6e6026b182e\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") "
Oct 30 15:57:53.753944 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.753922    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4067e045-ff27-4da7-99e3-e6e6026b182e-kube-api-access-5w4hj\" (UniqueName: \"kubernetes.io/projected/4067e045-ff27-4da7-99e3-e6e6026b182e-kube-api-access-5w4hj\") pod \"4067e045-ff27-4da7-99e3-e6e6026b182e\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") "
Oct 30 15:57:53.754083 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.753955    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4067e045-ff27-4da7-99e3-e6e6026b182e-tls-certs\" (UniqueName: \"kubernetes.io/secret/4067e045-ff27-4da7-99e3-e6e6026b182e-tls-certs\") pod \"4067e045-ff27-4da7-99e3-e6e6026b182e\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") "
Oct 30 15:57:53.754083 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.753995    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-dshm\") pod \"4067e045-ff27-4da7-99e3-e6e6026b182e\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") "
Oct 30 15:57:53.754083 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.754020    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-home" pod "4067e045-ff27-4da7-99e3-e6e6026b182e" (UID: "4067e045-ff27-4da7-99e3-e6e6026b182e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:53.754083 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.754039    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-model-cache\") pod \"4067e045-ff27-4da7-99e3-e6e6026b182e\" (UID: \"4067e045-ff27-4da7-99e3-e6e6026b182e\") "
Oct 30 15:57:53.755100 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.754756    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-model-cache" pod "4067e045-ff27-4da7-99e3-e6e6026b182e" (UID: "4067e045-ff27-4da7-99e3-e6e6026b182e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:53.755100 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.755057    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-tmp-dir" pod "4067e045-ff27-4da7-99e3-e6e6026b182e" (UID: "4067e045-ff27-4da7-99e3-e6e6026b182e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:53.755100 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.755081    4585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-model-cache\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:53.755277 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.755114    4585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-home\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:53.757828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.757399    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-dshm" pod "4067e045-ff27-4da7-99e3-e6e6026b182e" (UID: "4067e045-ff27-4da7-99e3-e6e6026b182e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:57:53.757828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.757452    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4067e045-ff27-4da7-99e3-e6e6026b182e-kube-api-access-5w4hj" pod "4067e045-ff27-4da7-99e3-e6e6026b182e" (UID: "4067e045-ff27-4da7-99e3-e6e6026b182e"). InnerVolumeSpecName "kube-api-access-5w4hj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:57:53.758185 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.758163    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4067e045-ff27-4da7-99e3-e6e6026b182e-tls-certs" pod "4067e045-ff27-4da7-99e3-e6e6026b182e" (UID: "4067e045-ff27-4da7-99e3-e6e6026b182e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:57:53.860376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.856721    4585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-tmp-dir\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:53.860376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.856758    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w4hj\" (UniqueName: \"kubernetes.io/projected/4067e045-ff27-4da7-99e3-e6e6026b182e-kube-api-access-5w4hj\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:53.860376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.856771    4585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4067e045-ff27-4da7-99e3-e6e6026b182e-tls-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:53.860376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.856784    4585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4067e045-ff27-4da7-99e3-e6e6026b182e-dshm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:53.992185 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.992100    4585 generic.go:356] "Generic (PLEG): container finished" podID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerID="f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116" exitCode=0
Oct 30 15:57:53.992185 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.992146    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"
Oct 30 15:57:53.992692 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.992188    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" event={"ID":"4067e045-ff27-4da7-99e3-e6e6026b182e","Type":"ContainerDied","Data":"f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116"}
Oct 30 15:57:53.992692 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.992229    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl" event={"ID":"4067e045-ff27-4da7-99e3-e6e6026b182e","Type":"ContainerDied","Data":"f811cddde00b1e668783c20490735ef6c31c792d6e34911f4d248b7c2f0827d8"}
Oct 30 15:57:53.992692 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:53.992251    4585 scope.go:122] "RemoveContainer" containerID="f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116"
Oct 30 15:57:54.004156 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:54.004024    4585 scope.go:122] "RemoveContainer" containerID="f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116"
Oct 30 15:57:54.004649 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:57:54.004569    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116\": container with ID starting with f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116 not found: ID does not exist" containerID="f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116"
Oct 30 15:57:54.004755 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:54.004639    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116"} err="failed to get container status \"f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116\": rpc error: code = NotFound desc = could not find container \"f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116\": container with ID starting with f247428f3d495d4438a522c3f056a36c2ca6d2f60e83519b12aa3ec02d172116 not found: ID does not exist"
Oct 30 15:57:54.049151 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:54.049121    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"]
Oct 30 15:57:54.054009 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:54.053976    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-c129d1/distinct-2-w6-c129d1-kserve-784b6476b6-5gttl"]
Oct 30 15:57:54.702275 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:54.702243    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" path="/var/lib/kubelet/pods/4067e045-ff27-4da7-99e3-e6e6026b182e/volumes"
Oct 30 15:57:54.903631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:54.903587    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"]
Oct 30 15:57:54.903865 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:54.903834    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb" podUID="c9d8649f-413e-486f-9499-aac47e0d7723" containerName="maas-api" containerID="cri-o://d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58" gracePeriod=30
Oct 30 15:57:55.785855 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:55.785817    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb" podUID="c9d8649f-413e-486f-9499-aac47e0d7723" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.45:8443/health\": dial tcp 10.134.0.45:8443: connect: connection refused"
Oct 30 15:57:56.171927 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.171899    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:57:56.199817 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.199787    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-metrics-certs\") pod \"c9d8649f-413e-486f-9499-aac47e0d7723\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") "
Oct 30 15:57:56.199982 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.199847    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-maas-api-tls\") pod \"c9d8649f-413e-486f-9499-aac47e0d7723\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") "
Oct 30 15:57:56.199982 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.199936    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9d8649f-413e-486f-9499-aac47e0d7723-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9d8649f-413e-486f-9499-aac47e0d7723-openshift-service-ca\") pod \"c9d8649f-413e-486f-9499-aac47e0d7723\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") "
Oct 30 15:57:56.200065 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.200021    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9d8649f-413e-486f-9499-aac47e0d7723-kube-api-access-cdxtr\" (UniqueName: \"kubernetes.io/projected/c9d8649f-413e-486f-9499-aac47e0d7723-kube-api-access-cdxtr\") pod \"c9d8649f-413e-486f-9499-aac47e0d7723\" (UID: \"c9d8649f-413e-486f-9499-aac47e0d7723\") "
Oct 30 15:57:56.200301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.200272    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9d8649f-413e-486f-9499-aac47e0d7723-openshift-service-ca" pod "c9d8649f-413e-486f-9499-aac47e0d7723" (UID: "c9d8649f-413e-486f-9499-aac47e0d7723"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:57:56.200593 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.200570    4585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9d8649f-413e-486f-9499-aac47e0d7723-openshift-service-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:56.202036 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.202012    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-metrics-certs" pod "c9d8649f-413e-486f-9499-aac47e0d7723" (UID: "c9d8649f-413e-486f-9499-aac47e0d7723"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:57:56.202151 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.202126    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-maas-api-tls" pod "c9d8649f-413e-486f-9499-aac47e0d7723" (UID: "c9d8649f-413e-486f-9499-aac47e0d7723"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:57:56.202223 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.202205    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9d8649f-413e-486f-9499-aac47e0d7723-kube-api-access-cdxtr" pod "c9d8649f-413e-486f-9499-aac47e0d7723" (UID: "c9d8649f-413e-486f-9499-aac47e0d7723"). InnerVolumeSpecName "kube-api-access-cdxtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:57:56.301913 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.301876    4585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-metrics-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:56.301913 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.301911    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9d8649f-413e-486f-9499-aac47e0d7723-maas-api-tls\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:56.301913 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.301922    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdxtr\" (UniqueName: \"kubernetes.io/projected/c9d8649f-413e-486f-9499-aac47e0d7723-kube-api-access-cdxtr\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:57:56.729499 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:56.729462    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb" podUID="d18ea2be-fa5a-45c8-8589-e081450576d4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 15:57:57.014537 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:57.014439    4585 generic.go:356] "Generic (PLEG): container finished" podID="c9d8649f-413e-486f-9499-aac47e0d7723" containerID="d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58" exitCode=0
Oct 30 15:57:57.014537 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:57.014498    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"
Oct 30 15:57:57.014537 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:57.014529    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb" event={"ID":"c9d8649f-413e-486f-9499-aac47e0d7723","Type":"ContainerDied","Data":"d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58"}
Oct 30 15:57:57.015126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:57.014589    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb" event={"ID":"c9d8649f-413e-486f-9499-aac47e0d7723","Type":"ContainerDied","Data":"02c37647bf06bd2ba7e9f78d676255ed7d6f280ce58154a2316c19f5a6f4c30e"}
Oct 30 15:57:57.015126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:57.014624    4585 scope.go:122] "RemoveContainer" containerID="d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58"
Oct 30 15:57:57.025245 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:57.025102    4585 scope.go:122] "RemoveContainer" containerID="d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58"
Oct 30 15:57:57.025432 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:57:57.025412    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58\": container with ID starting with d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58 not found: ID does not exist" containerID="d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58"
Oct 30 15:57:57.025470 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:57.025442    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58"} err="failed to get container status \"d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58\": rpc error: code = NotFound desc = could not find container \"d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58\": container with ID starting with d58517ba35d9c1898781a0ee5df184de5a967e5b14a344d2734980dc662e6b58 not found: ID does not exist"
Oct 30 15:57:57.066822 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:57.066790    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"]
Oct 30 15:57:57.079298 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:57.079271    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b63c3c-6d98c6fbf6-5rjtb"]
Oct 30 15:57:57.706714 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:57.706676    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf" podUID="133ba3b4-6cf3-42f1-a8ed-fa8285df655b" containerName="payload-processing" containerID="cri-o://772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0" gracePeriod=25
Oct 30 15:57:58.703147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:58.703102    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9d8649f-413e-486f-9499-aac47e0d7723" path="/var/lib/kubelet/pods/c9d8649f-413e-486f-9499-aac47e0d7723/volumes"
Oct 30 15:57:59.467552 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:59.467509    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"]
Oct 30 15:57:59.468034 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:57:59.467996    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv" podUID="0f1199e1-3446-4e9c-acca-96e7572c1952" containerName="maas-api" containerID="cri-o://2dc2de6b4590b2864599571e88c099376005c2fdb43b659f12c243ff8a03db10" gracePeriod=30
Oct 30 15:58:00.090427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.089754    4585 generic.go:356] "Generic (PLEG): container finished" podID="0f1199e1-3446-4e9c-acca-96e7572c1952" containerID="2dc2de6b4590b2864599571e88c099376005c2fdb43b659f12c243ff8a03db10" exitCode=0
Oct 30 15:58:00.090427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.089880    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv" event={"ID":"0f1199e1-3446-4e9c-acca-96e7572c1952","Type":"ContainerDied","Data":"2dc2de6b4590b2864599571e88c099376005c2fdb43b659f12c243ff8a03db10"}
Oct 30 15:58:00.179814 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.179779    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:58:00.282711 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.282647    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f1199e1-3446-4e9c-acca-96e7572c1952-kube-api-access-scz4g\" (UniqueName: \"kubernetes.io/projected/0f1199e1-3446-4e9c-acca-96e7572c1952-kube-api-access-scz4g\") pod \"0f1199e1-3446-4e9c-acca-96e7572c1952\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") "
Oct 30 15:58:00.282978 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.282781    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f1199e1-3446-4e9c-acca-96e7572c1952-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f1199e1-3446-4e9c-acca-96e7572c1952-openshift-service-ca\") pod \"0f1199e1-3446-4e9c-acca-96e7572c1952\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") "
Oct 30 15:58:00.283204 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.283180    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f1199e1-3446-4e9c-acca-96e7572c1952-openshift-service-ca" pod "0f1199e1-3446-4e9c-acca-96e7572c1952" (UID: "0f1199e1-3446-4e9c-acca-96e7572c1952"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:00.283477 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.283452    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-maas-api-tls\") pod \"0f1199e1-3446-4e9c-acca-96e7572c1952\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") "
Oct 30 15:58:00.283576 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.283514    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-metrics-certs\") pod \"0f1199e1-3446-4e9c-acca-96e7572c1952\" (UID: \"0f1199e1-3446-4e9c-acca-96e7572c1952\") "
Oct 30 15:58:00.284279 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.284124    4585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f1199e1-3446-4e9c-acca-96e7572c1952-openshift-service-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:00.285466 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.285437    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f1199e1-3446-4e9c-acca-96e7572c1952-kube-api-access-scz4g" pod "0f1199e1-3446-4e9c-acca-96e7572c1952" (UID: "0f1199e1-3446-4e9c-acca-96e7572c1952"). InnerVolumeSpecName "kube-api-access-scz4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:00.286277 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.286253    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-metrics-certs" pod "0f1199e1-3446-4e9c-acca-96e7572c1952" (UID: "0f1199e1-3446-4e9c-acca-96e7572c1952"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:58:00.296947 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.296920    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-maas-api-tls" pod "0f1199e1-3446-4e9c-acca-96e7572c1952" (UID: "0f1199e1-3446-4e9c-acca-96e7572c1952"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:58:00.386099 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.386017    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-maas-api-tls\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:00.386099 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.386047    4585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f1199e1-3446-4e9c-acca-96e7572c1952-metrics-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:00.386099 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.386057    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scz4g\" (UniqueName: \"kubernetes.io/projected/0f1199e1-3446-4e9c-acca-96e7572c1952-kube-api-access-scz4g\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:00.866126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:00.866087    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main" containerID="cri-o://ac9ae7a1fcc40b4a92ccb1810776255cf2d0aa65fad79ae78a282c13eed710d7" gracePeriod=45
Oct 30 15:58:01.099295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.099264    4585 generic.go:356] "Generic (PLEG): container finished" podID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerID="ac9ae7a1fcc40b4a92ccb1810776255cf2d0aa65fad79ae78a282c13eed710d7" exitCode=0
Oct 30 15:58:01.099821 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.099337    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" event={"ID":"e7821afe-257e-4224-82f4-2b365bfe5dc6","Type":"ContainerDied","Data":"ac9ae7a1fcc40b4a92ccb1810776255cf2d0aa65fad79ae78a282c13eed710d7"}
Oct 30 15:58:01.104375 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.104328    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv" event={"ID":"0f1199e1-3446-4e9c-acca-96e7572c1952","Type":"ContainerDied","Data":"a382a49cbf5890648314dc09a53a03e6bd72cda99203a8d8f6bb0e6903f17d09"}
Oct 30 15:58:01.104582 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.104560    4585 scope.go:122] "RemoveContainer" containerID="2dc2de6b4590b2864599571e88c099376005c2fdb43b659f12c243ff8a03db10"
Oct 30 15:58:01.104743 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.104729    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"
Oct 30 15:58:01.156825 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.156798    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:58:01.157400 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.157377    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"]
Oct 30 15:58:01.170727 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.170701    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-c129d1-876d8dbf6-zpmnv"]
Oct 30 15:58:01.299225 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.299105    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7821afe-257e-4224-82f4-2b365bfe5dc6-kube-api-access-kmwhr\" (UniqueName: \"kubernetes.io/projected/e7821afe-257e-4224-82f4-2b365bfe5dc6-kube-api-access-kmwhr\") pod \"e7821afe-257e-4224-82f4-2b365bfe5dc6\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") "
Oct 30 15:58:01.299225 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.299213    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-dshm\") pod \"e7821afe-257e-4224-82f4-2b365bfe5dc6\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") "
Oct 30 15:58:01.299478 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.299244    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-home\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-home\") pod \"e7821afe-257e-4224-82f4-2b365bfe5dc6\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") "
Oct 30 15:58:01.299478 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.299283    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-model-cache\") pod \"e7821afe-257e-4224-82f4-2b365bfe5dc6\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") "
Oct 30 15:58:01.299580 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.299505    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7821afe-257e-4224-82f4-2b365bfe5dc6-tls-certs\" (UniqueName: \"kubernetes.io/secret/e7821afe-257e-4224-82f4-2b365bfe5dc6-tls-certs\") pod \"e7821afe-257e-4224-82f4-2b365bfe5dc6\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") "
Oct 30 15:58:01.299632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.299607    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-tmp-dir\") pod \"e7821afe-257e-4224-82f4-2b365bfe5dc6\" (UID: \"e7821afe-257e-4224-82f4-2b365bfe5dc6\") "
Oct 30 15:58:01.300960 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.300926    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-tmp-dir" pod "e7821afe-257e-4224-82f4-2b365bfe5dc6" (UID: "e7821afe-257e-4224-82f4-2b365bfe5dc6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:01.302761 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.302731    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-home" pod "e7821afe-257e-4224-82f4-2b365bfe5dc6" (UID: "e7821afe-257e-4224-82f4-2b365bfe5dc6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:01.303020 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.302997    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-model-cache" pod "e7821afe-257e-4224-82f4-2b365bfe5dc6" (UID: "e7821afe-257e-4224-82f4-2b365bfe5dc6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:01.306684 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.306469    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-dshm" pod "e7821afe-257e-4224-82f4-2b365bfe5dc6" (UID: "e7821afe-257e-4224-82f4-2b365bfe5dc6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:01.310746 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.309966    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7821afe-257e-4224-82f4-2b365bfe5dc6-kube-api-access-kmwhr" pod "e7821afe-257e-4224-82f4-2b365bfe5dc6" (UID: "e7821afe-257e-4224-82f4-2b365bfe5dc6"). InnerVolumeSpecName "kube-api-access-kmwhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:01.312596 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.312558    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7821afe-257e-4224-82f4-2b365bfe5dc6-tls-certs" pod "e7821afe-257e-4224-82f4-2b365bfe5dc6" (UID: "e7821afe-257e-4224-82f4-2b365bfe5dc6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:58:01.401868 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.401827    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmwhr\" (UniqueName: \"kubernetes.io/projected/e7821afe-257e-4224-82f4-2b365bfe5dc6-kube-api-access-kmwhr\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:01.401868 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.401866    4585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-dshm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:01.402071 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.401881    4585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-home\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:01.402071 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.401894    4585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-model-cache\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:01.402071 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.401914    4585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7821afe-257e-4224-82f4-2b365bfe5dc6-tls-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:01.402071 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:01.401926    4585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7821afe-257e-4224-82f4-2b365bfe5dc6-tmp-dir\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:02.112162 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:02.112122    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"
Oct 30 15:58:02.112669 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:02.112122    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs" event={"ID":"e7821afe-257e-4224-82f4-2b365bfe5dc6","Type":"ContainerDied","Data":"754bf9bcc4db6a969845872606b52d4d94ea2d8f5f54793a719f984b39669af5"}
Oct 30 15:58:02.112669 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:02.112280    4585 scope.go:122] "RemoveContainer" containerID="ac9ae7a1fcc40b4a92ccb1810776255cf2d0aa65fad79ae78a282c13eed710d7"
Oct 30 15:58:02.165572 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:02.165534    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"]
Oct 30 15:58:02.177802 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:02.177773    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b24e7b/premium-w4-b24e7b-kserve-6b8957c6fb-rzwvs"]
Oct 30 15:58:02.703623 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:02.703590    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f1199e1-3446-4e9c-acca-96e7572c1952" path="/var/lib/kubelet/pods/0f1199e1-3446-4e9c-acca-96e7572c1952/volumes"
Oct 30 15:58:02.704081 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:02.704068    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" path="/var/lib/kubelet/pods/e7821afe-257e-4224-82f4-2b365bfe5dc6/volumes"
Oct 30 15:58:03.588567 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:03.588528    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"]
Oct 30 15:58:03.589116 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:03.588785    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy" containerID="cri-o://79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360" gracePeriod=30
Oct 30 15:58:03.593095 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:03.593064    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"]
Oct 30 15:58:03.593337 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:03.593304    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy" containerID="cri-o://a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed" gracePeriod=30
Oct 30 15:58:03.610126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:03.609699    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"]
Oct 30 15:58:03.610126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:03.609944    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy" containerID="cri-o://01c1f11005d4e9e6afbffc4b6011cf187e8034ed9d3a0abf152462d5529a9559" gracePeriod=30
Oct 30 15:58:03.614922 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:03.614895    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"]
Oct 30 15:58:03.615169 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:03.615122    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy" containerID="cri-o://1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3" gracePeriod=30
Oct 30 15:58:04.142949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:04.142912    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 15:58:04.143117 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:04.142971    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 15:58:04.663955 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:04.663918    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 15:58:04.664433 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:04.663982    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 15:58:05.192701 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:05.192666    4585 patch_prober.go:29] interesting pod/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 15:58:05.192873 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:05.192719    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 15:58:05.720654 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:05.720619    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf" podUID="133ba3b4-6cf3-42f1-a8ed-fa8285df655b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 15:58:06.728823 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:06.728781    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb" podUID="d18ea2be-fa5a-45c8-8589-e081450576d4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 15:58:07.047393 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.047342    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:58:07.149791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.149752    4585 generic.go:356] "Generic (PLEG): container finished" podID="d18ea2be-fa5a-45c8-8589-e081450576d4" containerID="945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0" exitCode=0
Oct 30 15:58:07.149958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.149801    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb" event={"ID":"d18ea2be-fa5a-45c8-8589-e081450576d4","Type":"ContainerDied","Data":"945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0"}
Oct 30 15:58:07.149958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.149881    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb" event={"ID":"d18ea2be-fa5a-45c8-8589-e081450576d4","Type":"ContainerDied","Data":"a76e69b9a545c387f27ff86b366d52e15805c41b307c5cb3ceb725e01446e2a4"}
Oct 30 15:58:07.149958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.149813    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"
Oct 30 15:58:07.149958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.149902    4585 scope.go:122] "RemoveContainer" containerID="945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0"
Oct 30 15:58:07.161782 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.161758    4585 scope.go:122] "RemoveContainer" containerID="945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0"
Oct 30 15:58:07.162131 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:07.162109    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0\": container with ID starting with 945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0 not found: ID does not exist" containerID="945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0"
Oct 30 15:58:07.162179 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.162140    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0"} err="failed to get container status \"945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0\": rpc error: code = NotFound desc = could not find container \"945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0\": container with ID starting with 945f1f73318d0104935367496c7b56591dca5aa237c48644d64730ae87792fd0 not found: ID does not exist"
Oct 30 15:58:07.216760 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.216729    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d18ea2be-fa5a-45c8-8589-e081450576d4-kube-api-access-zgdnx\" (UniqueName: \"kubernetes.io/projected/d18ea2be-fa5a-45c8-8589-e081450576d4-kube-api-access-zgdnx\") pod \"d18ea2be-fa5a-45c8-8589-e081450576d4\" (UID: \"d18ea2be-fa5a-45c8-8589-e081450576d4\") "
Oct 30 15:58:07.216944 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.216766    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d18ea2be-fa5a-45c8-8589-e081450576d4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d18ea2be-fa5a-45c8-8589-e081450576d4-plugins-config-volume\") pod \"d18ea2be-fa5a-45c8-8589-e081450576d4\" (UID: \"d18ea2be-fa5a-45c8-8589-e081450576d4\") "
Oct 30 15:58:07.217250 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.217222    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d18ea2be-fa5a-45c8-8589-e081450576d4-plugins-config-volume" pod "d18ea2be-fa5a-45c8-8589-e081450576d4" (UID: "d18ea2be-fa5a-45c8-8589-e081450576d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:07.217468 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.217452    4585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d18ea2be-fa5a-45c8-8589-e081450576d4-plugins-config-volume\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:07.219152 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.219126    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d18ea2be-fa5a-45c8-8589-e081450576d4-kube-api-access-zgdnx" pod "d18ea2be-fa5a-45c8-8589-e081450576d4" (UID: "d18ea2be-fa5a-45c8-8589-e081450576d4"). InnerVolumeSpecName "kube-api-access-zgdnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:07.318567 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.318489    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgdnx\" (UniqueName: \"kubernetes.io/projected/d18ea2be-fa5a-45c8-8589-e081450576d4-kube-api-access-zgdnx\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:07.496644 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.496614    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"]
Oct 30 15:58:07.501287 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:07.501260    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-c129d1-99b4bf859-k5hqb"]
Oct 30 15:58:08.165632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.165600    4585 generic.go:356] "Generic (PLEG): container finished" podID="8fdee16a-c007-40f8-9337-c896bc2c644e" containerID="590e4779660784159d2f6f0e1e4d92885eb95667e3316d9d02728dab257a0b63" exitCode=0
Oct 30 15:58:08.166106 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.165680    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-89g6q-gkgrp" event={"ID":"8fdee16a-c007-40f8-9337-c896bc2c644e","Type":"ContainerDied","Data":"590e4779660784159d2f6f0e1e4d92885eb95667e3316d9d02728dab257a0b63"}
Oct 30 15:58:08.298080 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.298045    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"]
Oct 30 15:58:08.306368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.298498    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy" containerID="cri-o://c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27" gracePeriod=30
Oct 30 15:58:08.352182 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.352145    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"]
Oct 30 15:58:08.356484 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.352654    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy" containerID="cri-o://cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194" gracePeriod=30
Oct 30 15:58:08.384407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.383229    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"]
Oct 30 15:58:08.384407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.384273    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy" containerID="cri-o://76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad" gracePeriod=30
Oct 30 15:58:08.389248 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.389220    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"]
Oct 30 15:58:08.389484 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.389454    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy" containerID="cri-o://517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b" gracePeriod=30
Oct 30 15:58:08.702978 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:08.702948    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d18ea2be-fa5a-45c8-8589-e081450576d4" path="/var/lib/kubelet/pods/d18ea2be-fa5a-45c8-8589-e081450576d4/volumes"
Oct 30 15:58:09.081827 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.081482    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:58:09.165556 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.165521    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:58:09.171470 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.171387    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-kube-api-access-v59pd\" (UniqueName: \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-kube-api-access-v59pd\") pod \"f2c5fdab-3219-4561-a188-9ec859fb77fc\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") "
Oct 30 15:58:09.171955 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.171931    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-envoy\") pod \"f2c5fdab-3219-4561-a188-9ec859fb77fc\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") "
Oct 30 15:58:09.172120 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.172105    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-podinfo\") pod \"f2c5fdab-3219-4561-a188-9ec859fb77fc\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") "
Oct 30 15:58:09.172247 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.172232    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2c5fdab-3219-4561-a188-9ec859fb77fc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2c5fdab-3219-4561-a188-9ec859fb77fc-istiod-ca-cert\") pod \"f2c5fdab-3219-4561-a188-9ec859fb77fc\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") "
Oct 30 15:58:09.172390 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.172368    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istiod-ca-cert\") pod \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") "
Oct 30 15:58:09.172513 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.172500    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-data\") pod \"f2c5fdab-3219-4561-a188-9ec859fb77fc\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") "
Oct 30 15:58:09.172636 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.172622    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-credential-socket\") pod \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") "
Oct 30 15:58:09.172734 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.172722    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-socket\") pod \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") "
Oct 30 15:58:09.172839 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.172826    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-data\") pod \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") "
Oct 30 15:58:09.173002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.172988    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-envoy\") pod \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") "
Oct 30 15:58:09.173197 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.173180    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-certs\") pod \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") "
Oct 30 15:58:09.173335 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.173318    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-credential-socket\") pod \"f2c5fdab-3219-4561-a188-9ec859fb77fc\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") "
Oct 30 15:58:09.173779 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.173762    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-token\" (UniqueName: \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-token\") pod \"f2c5fdab-3219-4561-a188-9ec859fb77fc\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") "
Oct 30 15:58:09.173926 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.173912    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-certs\") pod \"f2c5fdab-3219-4561-a188-9ec859fb77fc\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") "
Oct 30 15:58:09.174003 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.173994    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-token\" (UniqueName: \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-token\") pod \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") "
Oct 30 15:58:09.174087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.174077    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-kube-api-access-l82bc\" (UniqueName: \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-kube-api-access-l82bc\") pod \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") "
Oct 30 15:58:09.174168 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.174157    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-podinfo\") pod \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\" (UID: \"eafbf335-9c33-4b0b-92b6-da8f7b29d59f\") "
Oct 30 15:58:09.174243 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.174234    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-socket\") pod \"f2c5fdab-3219-4561-a188-9ec859fb77fc\" (UID: \"f2c5fdab-3219-4561-a188-9ec859fb77fc\") "
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.174764    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2c5fdab-3219-4561-a188-9ec859fb77fc-istiod-ca-cert" pod "f2c5fdab-3219-4561-a188-9ec859fb77fc" (UID: "f2c5fdab-3219-4561-a188-9ec859fb77fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.174997    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-socket" pod "f2c5fdab-3219-4561-a188-9ec859fb77fc" (UID: "f2c5fdab-3219-4561-a188-9ec859fb77fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.175039    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-socket" pod "eafbf335-9c33-4b0b-92b6-da8f7b29d59f" (UID: "eafbf335-9c33-4b0b-92b6-da8f7b29d59f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.175102    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-data" pod "f2c5fdab-3219-4561-a188-9ec859fb77fc" (UID: "f2c5fdab-3219-4561-a188-9ec859fb77fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.175199    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-data" pod "eafbf335-9c33-4b0b-92b6-da8f7b29d59f" (UID: "eafbf335-9c33-4b0b-92b6-da8f7b29d59f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.175502    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istiod-ca-cert" pod "eafbf335-9c33-4b0b-92b6-da8f7b29d59f" (UID: "eafbf335-9c33-4b0b-92b6-da8f7b29d59f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.175721    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-credential-socket" pod "eafbf335-9c33-4b0b-92b6-da8f7b29d59f" (UID: "eafbf335-9c33-4b0b-92b6-da8f7b29d59f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.176156    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2c5fdab-3219-4561-a188-9ec859fb77fc-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.176177    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.176189    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.176202    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.176218    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.176231    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.176245    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.176473    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-certs" pod "f2c5fdab-3219-4561-a188-9ec859fb77fc" (UID: "f2c5fdab-3219-4561-a188-9ec859fb77fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.176732    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-credential-socket" pod "f2c5fdab-3219-4561-a188-9ec859fb77fc" (UID: "f2c5fdab-3219-4561-a188-9ec859fb77fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.177147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.176918    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-certs" pod "eafbf335-9c33-4b0b-92b6-da8f7b29d59f" (UID: "eafbf335-9c33-4b0b-92b6-da8f7b29d59f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.178064 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.177769    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-envoy" pod "eafbf335-9c33-4b0b-92b6-da8f7b29d59f" (UID: "eafbf335-9c33-4b0b-92b6-da8f7b29d59f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.180631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.179598    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-envoy" pod "f2c5fdab-3219-4561-a188-9ec859fb77fc" (UID: "f2c5fdab-3219-4561-a188-9ec859fb77fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.180631 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.179961    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:58:09.182308 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.182280    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-podinfo" pod "eafbf335-9c33-4b0b-92b6-da8f7b29d59f" (UID: "eafbf335-9c33-4b0b-92b6-da8f7b29d59f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:09.186901 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.186875    4585 generic.go:356] "Generic (PLEG): container finished" podID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerID="a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed" exitCode=0
Oct 30 15:58:09.187467 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.187020    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" event={"ID":"eafbf335-9c33-4b0b-92b6-da8f7b29d59f","Type":"ContainerDied","Data":"a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed"}
Oct 30 15:58:09.187467 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.187079    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4" event={"ID":"eafbf335-9c33-4b0b-92b6-da8f7b29d59f","Type":"ContainerDied","Data":"b9def4c729e5a3be2d226dcda6505512fb9830fe909bb8b030c386b659477660"}
Oct 30 15:58:09.187467 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.187112    4585 scope.go:122] "RemoveContainer" containerID="a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed"
Oct 30 15:58:09.189469 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.189196    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-podinfo" pod "f2c5fdab-3219-4561-a188-9ec859fb77fc" (UID: "f2c5fdab-3219-4561-a188-9ec859fb77fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:09.189469 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.189277    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-token" pod "f2c5fdab-3219-4561-a188-9ec859fb77fc" (UID: "f2c5fdab-3219-4561-a188-9ec859fb77fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:09.189469 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.189416    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-kube-api-access-l82bc" pod "eafbf335-9c33-4b0b-92b6-da8f7b29d59f" (UID: "eafbf335-9c33-4b0b-92b6-da8f7b29d59f"). InnerVolumeSpecName "kube-api-access-l82bc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:09.189469 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.189454    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-token" pod "eafbf335-9c33-4b0b-92b6-da8f7b29d59f" (UID: "eafbf335-9c33-4b0b-92b6-da8f7b29d59f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:09.190459 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.190055    4585 generic.go:356] "Generic (PLEG): container finished" podID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerID="79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360" exitCode=0
Oct 30 15:58:09.190459 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.190142    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" event={"ID":"f2c5fdab-3219-4561-a188-9ec859fb77fc","Type":"ContainerDied","Data":"79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360"}
Oct 30 15:58:09.190459 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.190175    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp" event={"ID":"f2c5fdab-3219-4561-a188-9ec859fb77fc","Type":"ContainerDied","Data":"36102f713a5a82cfcd36e4b28aa69652eabb98dd97cda158b0ebf12d86d3714f"}
Oct 30 15:58:09.190459 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.190233    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"
Oct 30 15:58:09.193159 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.193052    4585 generic.go:356] "Generic (PLEG): container finished" podID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerID="01c1f11005d4e9e6afbffc4b6011cf187e8034ed9d3a0abf152462d5529a9559" exitCode=0
Oct 30 15:58:09.193159 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.193112    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" event={"ID":"35c54ae6-a861-4efc-8398-78b4610e0c40","Type":"ContainerDied","Data":"01c1f11005d4e9e6afbffc4b6011cf187e8034ed9d3a0abf152462d5529a9559"}
Oct 30 15:58:09.193928 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.193880    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-kube-api-access-v59pd" pod "f2c5fdab-3219-4561-a188-9ec859fb77fc" (UID: "f2c5fdab-3219-4561-a188-9ec859fb77fc"). InnerVolumeSpecName "kube-api-access-v59pd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:09.194077 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.194054    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"
Oct 30 15:58:09.196042 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.195957    4585 generic.go:356] "Generic (PLEG): container finished" podID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerID="1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3" exitCode=0
Oct 30 15:58:09.196042 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.196005    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"
Oct 30 15:58:09.196042 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.196018    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" event={"ID":"75ecb5a4-e09c-4d59-b943-ea660626247c","Type":"ContainerDied","Data":"1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3"}
Oct 30 15:58:09.196248 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.196072    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk" event={"ID":"75ecb5a4-e09c-4d59-b943-ea660626247c","Type":"ContainerDied","Data":"c65d729ec39b9542bb76cf12e900ac3175c342faca55c322e50fa4fdda3cf19e"}
Oct 30 15:58:09.207026 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.207008    4585 scope.go:122] "RemoveContainer" containerID="a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed"
Oct 30 15:58:09.207412 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:09.207335    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed\": container with ID starting with a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed not found: ID does not exist" containerID="a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed"
Oct 30 15:58:09.220732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.208632    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed"} err="failed to get container status \"a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed\": rpc error: code = NotFound desc = could not find container \"a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed\": container with ID starting with a02610dd642aad3b61129a135c172aa3be36dbb52e78dfbe9d92edb8dab550ed not found: ID does not exist"
Oct 30 15:58:09.220732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.208665    4585 scope.go:122] "RemoveContainer" containerID="79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360"
Oct 30 15:58:09.257407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.254543    4585 scope.go:122] "RemoveContainer" containerID="79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360"
Oct 30 15:58:09.265042 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:09.265006    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360\": container with ID starting with 79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360 not found: ID does not exist" containerID="79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360"
Oct 30 15:58:09.265239 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.265051    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360"} err="failed to get container status \"79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360\": rpc error: code = NotFound desc = could not find container \"79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360\": container with ID starting with 79b24679675ccd5b51c97d7949c7fec315f76fe5234a4ecd392850beb8229360 not found: ID does not exist"
Oct 30 15:58:09.265239 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.265078    4585 scope.go:122] "RemoveContainer" containerID="4b3eb7968795a66609551c011975cb8db7c07e132f339f1edb37636992114d30"
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.277007    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"]
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.277379    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-podinfo\") pod \"75ecb5a4-e09c-4d59-b943-ea660626247c\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") "
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.277436    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-socket\") pod \"75ecb5a4-e09c-4d59-b943-ea660626247c\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") "
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.277524    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-token\" (UniqueName: \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-token\") pod \"75ecb5a4-e09c-4d59-b943-ea660626247c\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") "
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.277561    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75ecb5a4-e09c-4d59-b943-ea660626247c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75ecb5a4-e09c-4d59-b943-ea660626247c-istiod-ca-cert\") pod \"75ecb5a4-e09c-4d59-b943-ea660626247c\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") "
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.277669    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-credential-socket\") pod \"75ecb5a4-e09c-4d59-b943-ea660626247c\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") "
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.277711    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-kube-api-access-qhg4l\" (UniqueName: \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-kube-api-access-qhg4l\") pod \"75ecb5a4-e09c-4d59-b943-ea660626247c\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") "
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.277763    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-data\") pod \"75ecb5a4-e09c-4d59-b943-ea660626247c\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") "
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.277886    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-certs\") pod \"75ecb5a4-e09c-4d59-b943-ea660626247c\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") "
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.277925    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-envoy\") pod \"75ecb5a4-e09c-4d59-b943-ea660626247c\" (UID: \"75ecb5a4-e09c-4d59-b943-ea660626247c\") "
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278488    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v59pd\" (UniqueName: \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-kube-api-access-v59pd\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278509    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278524    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278537    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278551    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278565    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278580    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2c5fdab-3219-4561-a188-9ec859fb77fc-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278595    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2c5fdab-3219-4561-a188-9ec859fb77fc-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278607    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278621    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l82bc\" (UniqueName: \"kubernetes.io/projected/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-kube-api-access-l82bc\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.278635    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eafbf335-9c33-4b0b-92b6-da8f7b29d59f-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.279275    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75ecb5a4-e09c-4d59-b943-ea660626247c-istiod-ca-cert" pod "75ecb5a4-e09c-4d59-b943-ea660626247c" (UID: "75ecb5a4-e09c-4d59-b943-ea660626247c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:09.279956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.279623    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-socket" pod "75ecb5a4-e09c-4d59-b943-ea660626247c" (UID: "75ecb5a4-e09c-4d59-b943-ea660626247c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.281763 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.281739    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-gdvs4"]
Oct 30 15:58:09.282005 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.281984    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-data" pod "75ecb5a4-e09c-4d59-b943-ea660626247c" (UID: "75ecb5a4-e09c-4d59-b943-ea660626247c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.282245 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.282223    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-certs" pod "75ecb5a4-e09c-4d59-b943-ea660626247c" (UID: "75ecb5a4-e09c-4d59-b943-ea660626247c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.288416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.287773    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-credential-socket" pod "75ecb5a4-e09c-4d59-b943-ea660626247c" (UID: "75ecb5a4-e09c-4d59-b943-ea660626247c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.290593 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.290535    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-token" pod "75ecb5a4-e09c-4d59-b943-ea660626247c" (UID: "75ecb5a4-e09c-4d59-b943-ea660626247c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:09.294912 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.293027    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:58:09.295714 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.295691    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-podinfo" pod "75ecb5a4-e09c-4d59-b943-ea660626247c" (UID: "75ecb5a4-e09c-4d59-b943-ea660626247c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:09.296209 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.296170    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-envoy" pod "75ecb5a4-e09c-4d59-b943-ea660626247c" (UID: "75ecb5a4-e09c-4d59-b943-ea660626247c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.303646 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.298515    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-kube-api-access-qhg4l" pod "75ecb5a4-e09c-4d59-b943-ea660626247c" (UID: "75ecb5a4-e09c-4d59-b943-ea660626247c"). InnerVolumeSpecName "kube-api-access-qhg4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:09.309210 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.309182    4585 scope.go:122] "RemoveContainer" containerID="1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3"
Oct 30 15:58:09.322207 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.322185    4585 scope.go:122] "RemoveContainer" containerID="1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3"
Oct 30 15:58:09.322652 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:09.322633    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3\": container with ID starting with 1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3 not found: ID does not exist" containerID="1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3"
Oct 30 15:58:09.322742 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.322660    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3"} err="failed to get container status \"1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3\": rpc error: code = NotFound desc = could not find container \"1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3\": container with ID starting with 1e5e4d3c0069302b994273ddc470e998cd2ba95818783167cf35a2bef7c83eb3 not found: ID does not exist"
Oct 30 15:58:09.379754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.379719    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-certs\") pod \"35c54ae6-a861-4efc-8398-78b4610e0c40\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") "
Oct 30 15:58:09.379927 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.379789    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-socket\") pod \"35c54ae6-a861-4efc-8398-78b4610e0c40\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") "
Oct 30 15:58:09.379927 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.379887    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-envoy\") pod \"35c54ae6-a861-4efc-8398-78b4610e0c40\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") "
Oct 30 15:58:09.380075 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.379950    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-kube-api-access-xxjr5\" (UniqueName: \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-kube-api-access-xxjr5\") pod \"35c54ae6-a861-4efc-8398-78b4610e0c40\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") "
Oct 30 15:58:09.380075 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.380013    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-podinfo\") pod \"35c54ae6-a861-4efc-8398-78b4610e0c40\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") "
Oct 30 15:58:09.380230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.380173    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-data\") pod \"35c54ae6-a861-4efc-8398-78b4610e0c40\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") "
Oct 30 15:58:09.380538 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.380507    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-credential-socket\") pod \"35c54ae6-a861-4efc-8398-78b4610e0c40\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") "
Oct 30 15:58:09.380689 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.380594    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-token\" (UniqueName: \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-token\") pod \"35c54ae6-a861-4efc-8398-78b4610e0c40\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") "
Oct 30 15:58:09.380689 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.380673    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35c54ae6-a861-4efc-8398-78b4610e0c40-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35c54ae6-a861-4efc-8398-78b4610e0c40-istiod-ca-cert\") pod \"35c54ae6-a861-4efc-8398-78b4610e0c40\" (UID: \"35c54ae6-a861-4efc-8398-78b4610e0c40\") "
Oct 30 15:58:09.381423 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.381400    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.381506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.381427    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.381506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.381440    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.381506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.381453    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.381506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.381466    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.381506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.381479    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.381506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.381492    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75ecb5a4-e09c-4d59-b943-ea660626247c-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.381506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.381505    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ecb5a4-e09c-4d59-b943-ea660626247c-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.381866 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.381518    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhg4l\" (UniqueName: \"kubernetes.io/projected/75ecb5a4-e09c-4d59-b943-ea660626247c-kube-api-access-qhg4l\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.381951 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.381900    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35c54ae6-a861-4efc-8398-78b4610e0c40-istiod-ca-cert" pod "35c54ae6-a861-4efc-8398-78b4610e0c40" (UID: "35c54ae6-a861-4efc-8398-78b4610e0c40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:09.382246 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.382222    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-certs" pod "35c54ae6-a861-4efc-8398-78b4610e0c40" (UID: "35c54ae6-a861-4efc-8398-78b4610e0c40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.382623 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.382436    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-socket" pod "35c54ae6-a861-4efc-8398-78b4610e0c40" (UID: "35c54ae6-a861-4efc-8398-78b4610e0c40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.382822 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.382786    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-data" pod "35c54ae6-a861-4efc-8398-78b4610e0c40" (UID: "35c54ae6-a861-4efc-8398-78b4610e0c40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.388980 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.388153    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-credential-socket" pod "35c54ae6-a861-4efc-8398-78b4610e0c40" (UID: "35c54ae6-a861-4efc-8398-78b4610e0c40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.390017 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.389975    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-envoy" pod "35c54ae6-a861-4efc-8398-78b4610e0c40" (UID: "35c54ae6-a861-4efc-8398-78b4610e0c40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.390017 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.389984    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-podinfo" pod "35c54ae6-a861-4efc-8398-78b4610e0c40" (UID: "35c54ae6-a861-4efc-8398-78b4610e0c40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:09.390188 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.390086    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-kube-api-access-xxjr5" pod "35c54ae6-a861-4efc-8398-78b4610e0c40" (UID: "35c54ae6-a861-4efc-8398-78b4610e0c40"). InnerVolumeSpecName "kube-api-access-xxjr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:09.390188 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.390134    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-token" pod "35c54ae6-a861-4efc-8398-78b4610e0c40" (UID: "35c54ae6-a861-4efc-8398-78b4610e0c40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:09.435908 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.435870    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:58:09.482578 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.482530    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-trusted-ca-bundle\") pod \"8fdee16a-c007-40f8-9337-c896bc2c644e\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") "
Oct 30 15:58:09.482740 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.482720    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-service-ca-bundle\") pod \"8fdee16a-c007-40f8-9337-c896bc2c644e\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") "
Oct 30 15:58:09.482832 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.482815    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fdee16a-c007-40f8-9337-c896bc2c644e-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8fdee16a-c007-40f8-9337-c896bc2c644e-archives-path\") pod \"8fdee16a-c007-40f8-9337-c896bc2c644e\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") "
Oct 30 15:58:09.482916 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.482898    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fdee16a-c007-40f8-9337-c896bc2c644e-kube-api-access-jftlz\" (UniqueName: \"kubernetes.io/projected/8fdee16a-c007-40f8-9337-c896bc2c644e-kube-api-access-jftlz\") pod \"8fdee16a-c007-40f8-9337-c896bc2c644e\" (UID: \"8fdee16a-c007-40f8-9337-c896bc2c644e\") "
Oct 30 15:58:09.483487 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.483465    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35c54ae6-a861-4efc-8398-78b4610e0c40-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.483570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.483492    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.483570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.483513    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.483570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.483527    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.483570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.483542    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxjr5\" (UniqueName: \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-kube-api-access-xxjr5\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.483570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.483555    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.483570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.483567    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.483895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.483580    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35c54ae6-a861-4efc-8398-78b4610e0c40-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.483895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.483592    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35c54ae6-a861-4efc-8398-78b4610e0c40-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.484295 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.484264    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fdee16a-c007-40f8-9337-c896bc2c644e-archives-path" pod "8fdee16a-c007-40f8-9337-c896bc2c644e" (UID: "8fdee16a-c007-40f8-9337-c896bc2c644e"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:09.484440 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.484307    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-service-ca-bundle" pod "8fdee16a-c007-40f8-9337-c896bc2c644e" (UID: "8fdee16a-c007-40f8-9337-c896bc2c644e"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:09.484440 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.484331    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-trusted-ca-bundle" pod "8fdee16a-c007-40f8-9337-c896bc2c644e" (UID: "8fdee16a-c007-40f8-9337-c896bc2c644e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:09.486724 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.486698    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fdee16a-c007-40f8-9337-c896bc2c644e-kube-api-access-jftlz" pod "8fdee16a-c007-40f8-9337-c896bc2c644e" (UID: "8fdee16a-c007-40f8-9337-c896bc2c644e"). InnerVolumeSpecName "kube-api-access-jftlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:09.543902 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.543866    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"]
Oct 30 15:58:09.549796 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.549765    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-vkwhp"]
Oct 30 15:58:09.582946 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.582911    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"]
Oct 30 15:58:09.584417 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.584389    4585 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-service-ca-bundle\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.584417 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.584412    4585 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/8fdee16a-c007-40f8-9337-c896bc2c644e-archives-path\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.584417 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.584423    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jftlz\" (UniqueName: \"kubernetes.io/projected/8fdee16a-c007-40f8-9337-c896bc2c644e-kube-api-access-jftlz\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.584581 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.584432    4585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fdee16a-c007-40f8-9337-c896bc2c644e-trusted-ca-bundle\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:09.586678 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:09.586657    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-np2qk"]
Oct 30 15:58:10.204420 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.204384    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-89g6q-gkgrp" event={"ID":"8fdee16a-c007-40f8-9337-c896bc2c644e","Type":"ContainerDied","Data":"d0a529fe930e7935de0f6201f07d9ea4c804d64abbc93c3851565b90f0404fa0"}
Oct 30 15:58:10.204829 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.204432    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-89g6q-gkgrp"
Oct 30 15:58:10.204829 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.204442    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0a529fe930e7935de0f6201f07d9ea4c804d64abbc93c3851565b90f0404fa0"
Oct 30 15:58:10.205876 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.205850    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"
Oct 30 15:58:10.205994 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.205852    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9" event={"ID":"35c54ae6-a861-4efc-8398-78b4610e0c40","Type":"ContainerDied","Data":"bb4f9519ce66b5d91252db5815c1765ee988ab7e7dce8b32707806374f880317"}
Oct 30 15:58:10.205994 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.205971    4585 scope.go:122] "RemoveContainer" containerID="01c1f11005d4e9e6afbffc4b6011cf187e8034ed9d3a0abf152462d5529a9559"
Oct 30 15:58:10.303554 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.303526    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"]
Oct 30 15:58:10.308341 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.308313    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b63c3c-openshift-default-5d7c6c46b5-95gb9"]
Oct 30 15:58:10.702341 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.702311    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" path="/var/lib/kubelet/pods/35c54ae6-a861-4efc-8398-78b4610e0c40/volumes"
Oct 30 15:58:10.702784 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.702768    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" path="/var/lib/kubelet/pods/75ecb5a4-e09c-4d59-b943-ea660626247c/volumes"
Oct 30 15:58:10.703183 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.703171    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" path="/var/lib/kubelet/pods/eafbf335-9c33-4b0b-92b6-da8f7b29d59f/volumes"
Oct 30 15:58:10.703836 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:10.703815    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" path="/var/lib/kubelet/pods/f2c5fdab-3219-4561-a188-9ec859fb77fc/volumes"
Oct 30 15:58:11.497201 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:11.497175    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:58:11.503602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:11.503582    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 15:58:13.698173 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.698137    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.748716    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-certs\") pod \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") "
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.748780    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-envoy\") pod \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") "
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.748912    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istiod-ca-cert\") pod \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") "
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.748957    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-token\" (UniqueName: \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-token\") pod \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") "
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.749069    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-credential-socket\") pod \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") "
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.749098    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-kube-api-access-r54qs\" (UniqueName: \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-kube-api-access-r54qs\") pod \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") "
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.749096    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-certs" pod "e6c8cdc3-f4e8-4a20-9015-423675fe5e55" (UID: "e6c8cdc3-f4e8-4a20-9015-423675fe5e55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.749155    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-podinfo\") pod \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") "
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.749210    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-socket\") pod \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") "
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.749236    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-data\") pod \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\" (UID: \"e6c8cdc3-f4e8-4a20-9015-423675fe5e55\") "
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.749554    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-credential-socket" pod "e6c8cdc3-f4e8-4a20-9015-423675fe5e55" (UID: "e6c8cdc3-f4e8-4a20-9015-423675fe5e55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.749740    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.749838 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.749755    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.750688 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.750496    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-socket" pod "e6c8cdc3-f4e8-4a20-9015-423675fe5e55" (UID: "e6c8cdc3-f4e8-4a20-9015-423675fe5e55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.752211 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.752180    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-envoy" pod "e6c8cdc3-f4e8-4a20-9015-423675fe5e55" (UID: "e6c8cdc3-f4e8-4a20-9015-423675fe5e55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.752803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.752774    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-kube-api-access-r54qs" pod "e6c8cdc3-f4e8-4a20-9015-423675fe5e55" (UID: "e6c8cdc3-f4e8-4a20-9015-423675fe5e55"). InnerVolumeSpecName "kube-api-access-r54qs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:13.753614 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.753585    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-data" pod "e6c8cdc3-f4e8-4a20-9015-423675fe5e55" (UID: "e6c8cdc3-f4e8-4a20-9015-423675fe5e55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.754512 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.754486    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istiod-ca-cert" pod "e6c8cdc3-f4e8-4a20-9015-423675fe5e55" (UID: "e6c8cdc3-f4e8-4a20-9015-423675fe5e55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:13.754763 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.754736    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-token" pod "e6c8cdc3-f4e8-4a20-9015-423675fe5e55" (UID: "e6c8cdc3-f4e8-4a20-9015-423675fe5e55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:13.754845 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.754810    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-podinfo" pod "e6c8cdc3-f4e8-4a20-9015-423675fe5e55" (UID: "e6c8cdc3-f4e8-4a20-9015-423675fe5e55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:13.763913 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.760251    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:58:13.768037 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.766975    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:58:13.850231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850186    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-podinfo\") pod \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") "
Oct 30 15:58:13.850460 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850249    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-envoy\") pod \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") "
Oct 30 15:58:13.850460 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850378    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00af3987-7922-49e4-a15f-9d4caeaa813d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00af3987-7922-49e4-a15f-9d4caeaa813d-istiod-ca-cert\") pod \"00af3987-7922-49e4-a15f-9d4caeaa813d\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") "
Oct 30 15:58:13.850460 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850413    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-podinfo\") pod \"00af3987-7922-49e4-a15f-9d4caeaa813d\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") "
Oct 30 15:58:13.850460 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850458    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-certs\") pod \"00af3987-7922-49e4-a15f-9d4caeaa813d\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") "
Oct 30 15:58:13.850721 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850497    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-kube-api-access-vsbwq\" (UniqueName: \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-kube-api-access-vsbwq\") pod \"00af3987-7922-49e4-a15f-9d4caeaa813d\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") "
Oct 30 15:58:13.850721 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850529    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-certs\") pod \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") "
Oct 30 15:58:13.850721 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850561    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istiod-ca-cert\") pod \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") "
Oct 30 15:58:13.850721 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850632    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-token\" (UniqueName: \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-token\") pod \"00af3987-7922-49e4-a15f-9d4caeaa813d\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") "
Oct 30 15:58:13.850721 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850664    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-data\") pod \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") "
Oct 30 15:58:13.850721 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850711    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-credential-socket\") pod \"00af3987-7922-49e4-a15f-9d4caeaa813d\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") "
Oct 30 15:58:13.851026 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850751    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-envoy\") pod \"00af3987-7922-49e4-a15f-9d4caeaa813d\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") "
Oct 30 15:58:13.851026 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850786    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-kube-api-access-dtt9t\" (UniqueName: \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-kube-api-access-dtt9t\") pod \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") "
Oct 30 15:58:13.851026 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850860    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-socket\") pod \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") "
Oct 30 15:58:13.851026 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850896    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-token\" (UniqueName: \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-token\") pod \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") "
Oct 30 15:58:13.851026 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.850938    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-data\") pod \"00af3987-7922-49e4-a15f-9d4caeaa813d\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") "
Oct 30 15:58:13.851026 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.851015    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-socket\") pod \"00af3987-7922-49e4-a15f-9d4caeaa813d\" (UID: \"00af3987-7922-49e4-a15f-9d4caeaa813d\") "
Oct 30 15:58:13.851325 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.851054    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-credential-socket\") pod \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\" (UID: \"fcb5f24d-b29c-4940-99cc-197fd3fde8ca\") "
Oct 30 15:58:13.851745 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.851720    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.851828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.851751    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.851828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.851766    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.851828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.851780    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r54qs\" (UniqueName: \"kubernetes.io/projected/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-kube-api-access-r54qs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.851828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.851795    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.851828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.851808    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.851828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.851822    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6c8cdc3-f4e8-4a20-9015-423675fe5e55-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.851828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.851818    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-certs" pod "fcb5f24d-b29c-4940-99cc-197fd3fde8ca" (UID: "fcb5f24d-b29c-4940-99cc-197fd3fde8ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.852193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.852002    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-credential-socket" pod "fcb5f24d-b29c-4940-99cc-197fd3fde8ca" (UID: "fcb5f24d-b29c-4940-99cc-197fd3fde8ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.852193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.852033    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-certs" pod "00af3987-7922-49e4-a15f-9d4caeaa813d" (UID: "00af3987-7922-49e4-a15f-9d4caeaa813d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.852472 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.852439    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-credential-socket" pod "00af3987-7922-49e4-a15f-9d4caeaa813d" (UID: "00af3987-7922-49e4-a15f-9d4caeaa813d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.852570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.852479    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00af3987-7922-49e4-a15f-9d4caeaa813d-istiod-ca-cert" pod "00af3987-7922-49e4-a15f-9d4caeaa813d" (UID: "00af3987-7922-49e4-a15f-9d4caeaa813d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:13.853482 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.853151    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-data" pod "fcb5f24d-b29c-4940-99cc-197fd3fde8ca" (UID: "fcb5f24d-b29c-4940-99cc-197fd3fde8ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.853482 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.853288    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-data" pod "00af3987-7922-49e4-a15f-9d4caeaa813d" (UID: "00af3987-7922-49e4-a15f-9d4caeaa813d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.853651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.853506    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-socket" pod "00af3987-7922-49e4-a15f-9d4caeaa813d" (UID: "00af3987-7922-49e4-a15f-9d4caeaa813d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.853713 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.853694    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-socket" pod "fcb5f24d-b29c-4940-99cc-197fd3fde8ca" (UID: "fcb5f24d-b29c-4940-99cc-197fd3fde8ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.854039 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.854014    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istiod-ca-cert" pod "fcb5f24d-b29c-4940-99cc-197fd3fde8ca" (UID: "fcb5f24d-b29c-4940-99cc-197fd3fde8ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:13.855082 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.855054    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-envoy" pod "00af3987-7922-49e4-a15f-9d4caeaa813d" (UID: "00af3987-7922-49e4-a15f-9d4caeaa813d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.862881 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.862846    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-token" pod "fcb5f24d-b29c-4940-99cc-197fd3fde8ca" (UID: "fcb5f24d-b29c-4940-99cc-197fd3fde8ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:13.863074 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.863044    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-kube-api-access-vsbwq" pod "00af3987-7922-49e4-a15f-9d4caeaa813d" (UID: "00af3987-7922-49e4-a15f-9d4caeaa813d"). InnerVolumeSpecName "kube-api-access-vsbwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:13.863643 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.863585    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-token" pod "00af3987-7922-49e4-a15f-9d4caeaa813d" (UID: "00af3987-7922-49e4-a15f-9d4caeaa813d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:13.863643 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.863605    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-envoy" pod "fcb5f24d-b29c-4940-99cc-197fd3fde8ca" (UID: "fcb5f24d-b29c-4940-99cc-197fd3fde8ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:13.863997 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.863964    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-podinfo" pod "00af3987-7922-49e4-a15f-9d4caeaa813d" (UID: "00af3987-7922-49e4-a15f-9d4caeaa813d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:13.866499 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.866456    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-kube-api-access-dtt9t" pod "fcb5f24d-b29c-4940-99cc-197fd3fde8ca" (UID: "fcb5f24d-b29c-4940-99cc-197fd3fde8ca"). InnerVolumeSpecName "kube-api-access-dtt9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:13.866981 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.866951    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-podinfo" pod "fcb5f24d-b29c-4940-99cc-197fd3fde8ca" (UID: "fcb5f24d-b29c-4940-99cc-197fd3fde8ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:13.956671 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956627    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00af3987-7922-49e4-a15f-9d4caeaa813d-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956671 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956671    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956685    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956702    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsbwq\" (UniqueName: \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-kube-api-access-vsbwq\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956718    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956731    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956744    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956757    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956769    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956781    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956795    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtt9t\" (UniqueName: \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-kube-api-access-dtt9t\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956809    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956822    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956834    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956847    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00af3987-7922-49e4-a15f-9d4caeaa813d-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956860    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956873    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:13.956923 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:13.956886    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb5f24d-b29c-4940-99cc-197fd3fde8ca-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:14.078301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.078262    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:58:14.159095 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159057    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-socket\") pod \"ba4c3f49-4422-483f-b24f-41545ed46667\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") "
Oct 30 15:58:14.159382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159113    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-kube-api-access-8bmlx\" (UniqueName: \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-kube-api-access-8bmlx\") pod \"ba4c3f49-4422-483f-b24f-41545ed46667\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") "
Oct 30 15:58:14.159382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159144    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-envoy\") pod \"ba4c3f49-4422-483f-b24f-41545ed46667\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") "
Oct 30 15:58:14.159382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159166    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ba4c3f49-4422-483f-b24f-41545ed46667-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba4c3f49-4422-483f-b24f-41545ed46667-istiod-ca-cert\") pod \"ba4c3f49-4422-483f-b24f-41545ed46667\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") "
Oct 30 15:58:14.159382 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159308    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-certs\") pod \"ba4c3f49-4422-483f-b24f-41545ed46667\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") "
Oct 30 15:58:14.159624 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159405    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ba4c3f49-4422-483f-b24f-41545ed46667-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba4c3f49-4422-483f-b24f-41545ed46667-istio-podinfo\") pod \"ba4c3f49-4422-483f-b24f-41545ed46667\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") "
Oct 30 15:58:14.159624 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159431    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-socket" pod "ba4c3f49-4422-483f-b24f-41545ed46667" (UID: "ba4c3f49-4422-483f-b24f-41545ed46667"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:14.159624 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159471    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-credential-socket\") pod \"ba4c3f49-4422-483f-b24f-41545ed46667\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") "
Oct 30 15:58:14.159794 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159771    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-credential-socket" pod "ba4c3f49-4422-483f-b24f-41545ed46667" (UID: "ba4c3f49-4422-483f-b24f-41545ed46667"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:14.159936 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159901    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-data\") pod \"ba4c3f49-4422-483f-b24f-41545ed46667\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") "
Oct 30 15:58:14.160020 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.159997    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-istio-token\" (UniqueName: \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-istio-token\") pod \"ba4c3f49-4422-483f-b24f-41545ed46667\" (UID: \"ba4c3f49-4422-483f-b24f-41545ed46667\") "
Oct 30 15:58:14.160390 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.160342    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba4c3f49-4422-483f-b24f-41545ed46667-istiod-ca-cert" pod "ba4c3f49-4422-483f-b24f-41545ed46667" (UID: "ba4c3f49-4422-483f-b24f-41545ed46667"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:14.160653 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.160632    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:14.160774 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.160758    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:14.160996 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.160979    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba4c3f49-4422-483f-b24f-41545ed46667-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:14.161127 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.160911    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-certs" pod "ba4c3f49-4422-483f-b24f-41545ed46667" (UID: "ba4c3f49-4422-483f-b24f-41545ed46667"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:14.161238 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.160950    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-data" pod "ba4c3f49-4422-483f-b24f-41545ed46667" (UID: "ba4c3f49-4422-483f-b24f-41545ed46667"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:14.162221 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.162190    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba4c3f49-4422-483f-b24f-41545ed46667-istio-podinfo" pod "ba4c3f49-4422-483f-b24f-41545ed46667" (UID: "ba4c3f49-4422-483f-b24f-41545ed46667"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:14.162339 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.162321    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-kube-api-access-8bmlx" pod "ba4c3f49-4422-483f-b24f-41545ed46667" (UID: "ba4c3f49-4422-483f-b24f-41545ed46667"). InnerVolumeSpecName "kube-api-access-8bmlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:14.163166 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.163146    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-istio-token" pod "ba4c3f49-4422-483f-b24f-41545ed46667" (UID: "ba4c3f49-4422-483f-b24f-41545ed46667"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:14.164603 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.164580    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-envoy" pod "ba4c3f49-4422-483f-b24f-41545ed46667" (UID: "ba4c3f49-4422-483f-b24f-41545ed46667"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:14.235196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.235158    4585 generic.go:356] "Generic (PLEG): container finished" podID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerID="76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad" exitCode=0
Oct 30 15:58:14.235436 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.235207    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"
Oct 30 15:58:14.235436 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.235259    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" event={"ID":"00af3987-7922-49e4-a15f-9d4caeaa813d","Type":"ContainerDied","Data":"76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad"}
Oct 30 15:58:14.235436 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.235372    4585 scope.go:122] "RemoveContainer" containerID="76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad"
Oct 30 15:58:14.235617 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.235338    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7" event={"ID":"00af3987-7922-49e4-a15f-9d4caeaa813d","Type":"ContainerDied","Data":"80fba14c56196a97c530ca3c3c48f30a89efb58aa10ab959ce48b4be3c8f94a5"}
Oct 30 15:58:14.246410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.243044    4585 generic.go:356] "Generic (PLEG): container finished" podID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerID="517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b" exitCode=0
Oct 30 15:58:14.246410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.243221    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" event={"ID":"fcb5f24d-b29c-4940-99cc-197fd3fde8ca","Type":"ContainerDied","Data":"517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b"}
Oct 30 15:58:14.246410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.243272    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47" event={"ID":"fcb5f24d-b29c-4940-99cc-197fd3fde8ca","Type":"ContainerDied","Data":"66532253f74eaf335306810e4df25d45ce44f34edb26050caa2b639a9ed3fdcc"}
Oct 30 15:58:14.246410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.243371    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"
Oct 30 15:58:14.263744 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.260339    4585 generic.go:356] "Generic (PLEG): container finished" podID="ba4c3f49-4422-483f-b24f-41545ed46667" containerID="c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27" exitCode=0
Oct 30 15:58:14.263744 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.260484    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" event={"ID":"ba4c3f49-4422-483f-b24f-41545ed46667","Type":"ContainerDied","Data":"c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27"}
Oct 30 15:58:14.263744 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.260532    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng" event={"ID":"ba4c3f49-4422-483f-b24f-41545ed46667","Type":"ContainerDied","Data":"6c650c45f2bcec31d5c31a0f0667a8078514d7f3fc839e210fa6830dd2b55b47"}
Oct 30 15:58:14.263744 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.260584    4585 scope.go:122] "RemoveContainer" containerID="76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad"
Oct 30 15:58:14.263744 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.260698    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.266332    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.266395    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bmlx\" (UniqueName: \"kubernetes.io/projected/ba4c3f49-4422-483f-b24f-41545ed46667-kube-api-access-8bmlx\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.266409    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.266422    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.266433    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba4c3f49-4422-483f-b24f-41545ed46667-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.266445    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba4c3f49-4422-483f-b24f-41545ed46667-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:14.265151    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad\": container with ID starting with 76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad not found: ID does not exist" containerID="76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad"
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.268436    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad"} err="failed to get container status \"76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad\": rpc error: code = NotFound desc = could not find container \"76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad\": container with ID starting with 76d267493502da7e42fa5a2ffb998bbea449d790d0897449201b0a7055eb52ad not found: ID does not exist"
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.268455    4585 scope.go:122] "RemoveContainer" containerID="517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b"
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.276536    4585 generic.go:356] "Generic (PLEG): container finished" podID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerID="cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194" exitCode=0
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.276672    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" event={"ID":"e6c8cdc3-f4e8-4a20-9015-423675fe5e55","Type":"ContainerDied","Data":"cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194"}
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.276722    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz" event={"ID":"e6c8cdc3-f4e8-4a20-9015-423675fe5e55","Type":"ContainerDied","Data":"283dc77340ede50bfa6e9cfd7903ce3f3c7f28668ee380d11e07b65c414386c9"}
Oct 30 15:58:14.286878 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.276800    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"
Oct 30 15:58:14.296218 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.296201    4585 scope.go:122] "RemoveContainer" containerID="517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b"
Oct 30 15:58:14.297992 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:14.296631    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b\": container with ID starting with 517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b not found: ID does not exist" containerID="517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b"
Oct 30 15:58:14.297992 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.296676    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b"} err="failed to get container status \"517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b\": rpc error: code = NotFound desc = could not find container \"517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b\": container with ID starting with 517dceacfa3d5806b024fac38c74d345c028a7a78d802aaf54bd43114c04f14b not found: ID does not exist"
Oct 30 15:58:14.297992 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.296694    4585 scope.go:122] "RemoveContainer" containerID="c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27"
Oct 30 15:58:14.316410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.316391    4585 scope.go:122] "RemoveContainer" containerID="971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143"
Oct 30 15:58:14.326670 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.326649    4585 scope.go:122] "RemoveContainer" containerID="c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27"
Oct 30 15:58:14.327036 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:14.327010    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27\": container with ID starting with c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27 not found: ID does not exist" containerID="c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27"
Oct 30 15:58:14.327103 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.327046    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27"} err="failed to get container status \"c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27\": rpc error: code = NotFound desc = could not find container \"c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27\": container with ID starting with c7213ad82e03987d947e0244ad086ab07052aa99d9e99df46786bf398b082f27 not found: ID does not exist"
Oct 30 15:58:14.327103 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.327063    4585 scope.go:122] "RemoveContainer" containerID="971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143"
Oct 30 15:58:14.327334 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:14.327316    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143\": container with ID starting with 971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143 not found: ID does not exist" containerID="971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143"
Oct 30 15:58:14.327425 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.327357    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143"} err="failed to get container status \"971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143\": rpc error: code = NotFound desc = could not find container \"971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143\": container with ID starting with 971ce29f798803e8ad7d5a2c0c8392744c93cd0ebebc540524d91fc6c855e143 not found: ID does not exist"
Oct 30 15:58:14.327425 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.327380    4585 scope.go:122] "RemoveContainer" containerID="cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194"
Oct 30 15:58:14.335875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.335847    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"]
Oct 30 15:58:14.338668 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.338645    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-6f7p7"]
Oct 30 15:58:14.343252 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.342257    4585 scope.go:122] "RemoveContainer" containerID="cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194"
Oct 30 15:58:14.343252 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:14.342708    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194\": container with ID starting with cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194 not found: ID does not exist" containerID="cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194"
Oct 30 15:58:14.343252 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.342741    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194"} err="failed to get container status \"cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194\": rpc error: code = NotFound desc = could not find container \"cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194\": container with ID starting with cedb0fd3ae1e47fcf09c5f1faa3fd901d4a8d19caca863234de1d6b3a5967194 not found: ID does not exist"
Oct 30 15:58:14.375939 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.375905    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"]
Oct 30 15:58:14.380819 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.380783    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-t9wng"]
Oct 30 15:58:14.414561 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.414470    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"]
Oct 30 15:58:14.428843 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.428810    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-8dntz"]
Oct 30 15:58:14.469153 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.469120    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"]
Oct 30 15:58:14.475936 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.475906    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c129d1-openshift-default-5b5796b49c-czc47"]
Oct 30 15:58:14.703945 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.703864    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" path="/var/lib/kubelet/pods/00af3987-7922-49e4-a15f-9d4caeaa813d/volumes"
Oct 30 15:58:14.704619 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.704597    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" path="/var/lib/kubelet/pods/ba4c3f49-4422-483f-b24f-41545ed46667/volumes"
Oct 30 15:58:14.705274 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.705255    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" path="/var/lib/kubelet/pods/e6c8cdc3-f4e8-4a20-9015-423675fe5e55/volumes"
Oct 30 15:58:14.706424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:14.706404    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" path="/var/lib/kubelet/pods/fcb5f24d-b29c-4940-99cc-197fd3fde8ca/volumes"
Oct 30 15:58:15.722262 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:15.722206    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf" podUID="133ba3b4-6cf3-42f1-a8ed-fa8285df655b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 15:58:16.040135 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:16.040032    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"]
Oct 30 15:58:16.050628 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:16.050591    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ede796e5-65bw4"]
Oct 30 15:58:16.763415 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:16.763378    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c243a881-5d94-4c4c-a234-fd092457363b" path="/var/lib/kubelet/pods/c243a881-5d94-4c4c-a234-fd092457363b/volumes"
Oct 30 15:58:17.288467 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:17.288426    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:58:18.013429 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:18.013286    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:58:18.052468 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.052434    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:58:18.182823 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.182754    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-plugins-config-volume\") pod \"133ba3b4-6cf3-42f1-a8ed-fa8285df655b\" (UID: \"133ba3b4-6cf3-42f1-a8ed-fa8285df655b\") "
Oct 30 15:58:18.183025 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.182881    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-kube-api-access-2f4nb\" (UniqueName: \"kubernetes.io/projected/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-kube-api-access-2f4nb\") pod \"133ba3b4-6cf3-42f1-a8ed-fa8285df655b\" (UID: \"133ba3b4-6cf3-42f1-a8ed-fa8285df655b\") "
Oct 30 15:58:18.188369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.184118    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-plugins-config-volume" pod "133ba3b4-6cf3-42f1-a8ed-fa8285df655b" (UID: "133ba3b4-6cf3-42f1-a8ed-fa8285df655b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:18.188369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.186716    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-kube-api-access-2f4nb" pod "133ba3b4-6cf3-42f1-a8ed-fa8285df655b" (UID: "133ba3b4-6cf3-42f1-a8ed-fa8285df655b"). InnerVolumeSpecName "kube-api-access-2f4nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:18.284425 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.284269    4585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-plugins-config-volume\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:18.284425 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.284308    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f4nb\" (UniqueName: \"kubernetes.io/projected/133ba3b4-6cf3-42f1-a8ed-fa8285df655b-kube-api-access-2f4nb\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:18.416036 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.416005    4585 generic.go:356] "Generic (PLEG): container finished" podID="133ba3b4-6cf3-42f1-a8ed-fa8285df655b" containerID="772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0" exitCode=0
Oct 30 15:58:18.416190 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.416049    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"
Oct 30 15:58:18.416190 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.416091    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf" event={"ID":"133ba3b4-6cf3-42f1-a8ed-fa8285df655b","Type":"ContainerDied","Data":"772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0"}
Oct 30 15:58:18.416190 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.416145    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf" event={"ID":"133ba3b4-6cf3-42f1-a8ed-fa8285df655b","Type":"ContainerDied","Data":"57a6b4c47e078e61b3c0f613e0d1790e4db479748d1d2f070680ec1f6b9ff905"}
Oct 30 15:58:18.416190 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.416168    4585 scope.go:122] "RemoveContainer" containerID="772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0"
Oct 30 15:58:18.425218 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.425202    4585 scope.go:122] "RemoveContainer" containerID="772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0"
Oct 30 15:58:18.425535 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:18.425517    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0\": container with ID starting with 772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0 not found: ID does not exist" containerID="772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0"
Oct 30 15:58:18.425632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.425545    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0"} err="failed to get container status \"772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0\": rpc error: code = NotFound desc = could not find container \"772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0\": container with ID starting with 772c836d249c99ec3c3f5efa712a5eba0096b6efe94378d572e39114fd759bd0 not found: ID does not exist"
Oct 30 15:58:18.479485 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.479452    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"]
Oct 30 15:58:18.490600 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.490568    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b24e7b-5db8c74566-tpswf"]
Oct 30 15:58:18.522464 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.522429    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"]
Oct 30 15:58:18.522958 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.522940    4585 cpu_manager.go:414] "containerMap: removing container" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main"
Oct 30 15:58:18.523063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.522960    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main"
Oct 30 15:58:18.523063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.522977    4585 cpu_manager.go:414] "containerMap: removing container" podUID="d18ea2be-fa5a-45c8-8589-e081450576d4" containerName="payload-pre-processing"
Oct 30 15:58:18.523063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.522985    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d18ea2be-fa5a-45c8-8589-e081450576d4" containerName="payload-pre-processing"
Oct 30 15:58:18.523063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.522996    4585 cpu_manager.go:414] "containerMap: removing container" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy"
Oct 30 15:58:18.523063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523003    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy"
Oct 30 15:58:18.523063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523019    4585 cpu_manager.go:414] "containerMap: removing container" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy"
Oct 30 15:58:18.523063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523026    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy"
Oct 30 15:58:18.523063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523041    4585 cpu_manager.go:414] "containerMap: removing container" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main"
Oct 30 15:58:18.523063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523048    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523067    4585 cpu_manager.go:414] "containerMap: removing container" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523075    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523085    4585 cpu_manager.go:414] "containerMap: removing container" podUID="c9d8649f-413e-486f-9499-aac47e0d7723" containerName="maas-api"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523093    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9d8649f-413e-486f-9499-aac47e0d7723" containerName="maas-api"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523104    4585 cpu_manager.go:414] "containerMap: removing container" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523111    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523132    4585 cpu_manager.go:414] "containerMap: removing container" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523139    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523164    4585 cpu_manager.go:414] "containerMap: removing container" podUID="0f1199e1-3446-4e9c-acca-96e7572c1952" containerName="maas-api"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523172    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f1199e1-3446-4e9c-acca-96e7572c1952" containerName="maas-api"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523182    4585 cpu_manager.go:414] "containerMap: removing container" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523189    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523200    4585 cpu_manager.go:414] "containerMap: removing container" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523207    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523221    4585 cpu_manager.go:414] "containerMap: removing container" podUID="133ba3b4-6cf3-42f1-a8ed-fa8285df655b" containerName="payload-processing"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523228    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="133ba3b4-6cf3-42f1-a8ed-fa8285df655b" containerName="payload-processing"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523238    4585 cpu_manager.go:414] "containerMap: removing container" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523245    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523256    4585 cpu_manager.go:414] "containerMap: removing container" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523263    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523272    4585 cpu_manager.go:414] "containerMap: removing container" podUID="8fdee16a-c007-40f8-9337-c896bc2c644e" containerName="insights-gathering"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523279    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8fdee16a-c007-40f8-9337-c896bc2c644e" containerName="insights-gathering"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523291    4585 cpu_manager.go:414] "containerMap: removing container" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523298    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523422    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="4067e045-ff27-4da7-99e3-e6e6026b182e" containerName="main"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523436    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="d18ea2be-fa5a-45c8-8589-e081450576d4" containerName="payload-pre-processing"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523446    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523458    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="75ecb5a4-e09c-4d59-b943-ea660626247c" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523467    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7821afe-257e-4224-82f4-2b365bfe5dc6" containerName="main"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523478    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="eafbf335-9c33-4b0b-92b6-da8f7b29d59f" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523490    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523498    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9d8649f-413e-486f-9499-aac47e0d7723" containerName="maas-api"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523505    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6c8cdc3-f4e8-4a20-9015-423675fe5e55" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523520    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f1199e1-3446-4e9c-acca-96e7572c1952" containerName="maas-api"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523529    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="00af3987-7922-49e4-a15f-9d4caeaa813d" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523540    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcb5f24d-b29c-4940-99cc-197fd3fde8ca" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523548    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="133ba3b4-6cf3-42f1-a8ed-fa8285df655b" containerName="payload-processing"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523555    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2c5fdab-3219-4561-a188-9ec859fb77fc" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523562    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523571    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfa4b03a-3d65-4fdf-99cc-704df480ca67" containerName="main"
Oct 30 15:58:18.523559 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523579    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="8fdee16a-c007-40f8-9337-c896bc2c644e" containerName="insights-gathering"
Oct 30 15:58:18.525236 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.523588    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0ae31d1-0e25-441b-8257-4871a18ed6ee" containerName="main"
Oct 30 15:58:18.528895 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.528874    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:18.544621 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.544595    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 15:58:18.545856 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.545830    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 15:58:18.546168 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.546150    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 15:58:18.546696 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.546675    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 15:58:18.547458 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.547434    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"]
Oct 30 15:58:18.588264 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.588220    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szb69\" (UniqueName: \"kubernetes.io/projected/96fb8d85-a90d-4c9f-9772-5da645f27721-kube-api-access-szb69\") pod \"maas-api-revoke-sub-cc62ed0f-fzxcz\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:18.588484 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.588313    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc62ed0f-fzxcz\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:18.588549 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.588528    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc62ed0f-fzxcz\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:18.689614 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.689574    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc62ed0f-fzxcz\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:18.689783 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.689645    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szb69\" (UniqueName: \"kubernetes.io/projected/96fb8d85-a90d-4c9f-9772-5da645f27721-kube-api-access-szb69\") pod \"maas-api-revoke-sub-cc62ed0f-fzxcz\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:18.689783 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.689680    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc62ed0f-fzxcz\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:18.690415 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.690390    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc62ed0f-fzxcz\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:18.692112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.692089    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc62ed0f-fzxcz\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:18.702257 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.702223    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="133ba3b4-6cf3-42f1-a8ed-fa8285df655b" path="/var/lib/kubelet/pods/133ba3b4-6cf3-42f1-a8ed-fa8285df655b/volumes"
Oct 30 15:58:18.713922 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.713888    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szb69\" (UniqueName: \"kubernetes.io/projected/96fb8d85-a90d-4c9f-9772-5da645f27721-kube-api-access-szb69\") pod \"maas-api-revoke-sub-cc62ed0f-fzxcz\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:18.842961 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:18.842878    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:19.187392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:19.186915    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"]
Oct 30 15:58:19.194001 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:58:19.193946    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96fb8d85_a90d_4c9f_9772_5da645f27721.slice/crio-eb0aa41a33e5c86ca3c611aedc802a856cece00e2ad0c53ecd19b847463a8400 WatchSource:0}: Error finding container eb0aa41a33e5c86ca3c611aedc802a856cece00e2ad0c53ecd19b847463a8400: Status 404 returned error can't find the container with id eb0aa41a33e5c86ca3c611aedc802a856cece00e2ad0c53ecd19b847463a8400
Oct 30 15:58:19.424508 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:19.424474    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz" event={"ID":"96fb8d85-a90d-4c9f-9772-5da645f27721","Type":"ContainerStarted","Data":"eb0aa41a33e5c86ca3c611aedc802a856cece00e2ad0c53ecd19b847463a8400"}
Oct 30 15:58:20.431968 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:20.431931    4585 generic.go:356] "Generic (PLEG): container finished" podID="96fb8d85-a90d-4c9f-9772-5da645f27721" containerID="bc95983449ddd9fce56c43f181cbeb885a953d952df5c8feee271047b3577ef5" exitCode=0
Oct 30 15:58:20.431968 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:20.431981    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz" event={"ID":"96fb8d85-a90d-4c9f-9772-5da645f27721","Type":"ContainerDied","Data":"bc95983449ddd9fce56c43f181cbeb885a953d952df5c8feee271047b3577ef5"}
Oct 30 15:58:21.620301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:21.620274    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:21.725582 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:21.725495    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-openshift-ca\") pod \"96fb8d85-a90d-4c9f-9772-5da645f27721\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") "
Oct 30 15:58:21.725726 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:21.725617    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-xks-ca\") pod \"96fb8d85-a90d-4c9f-9772-5da645f27721\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") "
Oct 30 15:58:21.725726 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:21.725672    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96fb8d85-a90d-4c9f-9772-5da645f27721-kube-api-access-szb69\" (UniqueName: \"kubernetes.io/projected/96fb8d85-a90d-4c9f-9772-5da645f27721-kube-api-access-szb69\") pod \"96fb8d85-a90d-4c9f-9772-5da645f27721\" (UID: \"96fb8d85-a90d-4c9f-9772-5da645f27721\") "
Oct 30 15:58:21.725955 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:21.725922    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-openshift-ca" pod "96fb8d85-a90d-4c9f-9772-5da645f27721" (UID: "96fb8d85-a90d-4c9f-9772-5da645f27721"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:21.726140 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:21.726125    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:21.727782 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:21.727753    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-xks-ca" pod "96fb8d85-a90d-4c9f-9772-5da645f27721" (UID: "96fb8d85-a90d-4c9f-9772-5da645f27721"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:58:21.727936 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:21.727913    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96fb8d85-a90d-4c9f-9772-5da645f27721-kube-api-access-szb69" pod "96fb8d85-a90d-4c9f-9772-5da645f27721" (UID: "96fb8d85-a90d-4c9f-9772-5da645f27721"). InnerVolumeSpecName "kube-api-access-szb69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:21.827771 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:21.827730    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96fb8d85-a90d-4c9f-9772-5da645f27721-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:21.827771 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:21.827768    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szb69\" (UniqueName: \"kubernetes.io/projected/96fb8d85-a90d-4c9f-9772-5da645f27721-kube-api-access-szb69\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:22.444459 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:22.444415    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz" event={"ID":"96fb8d85-a90d-4c9f-9772-5da645f27721","Type":"ContainerDied","Data":"eb0aa41a33e5c86ca3c611aedc802a856cece00e2ad0c53ecd19b847463a8400"}
Oct 30 15:58:22.444459 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:22.444447    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"
Oct 30 15:58:22.444671 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:22.444485    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb0aa41a33e5c86ca3c611aedc802a856cece00e2ad0c53ecd19b847463a8400"
Oct 30 15:58:27.830200 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.830111    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"]
Oct 30 15:58:27.830825 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.830805    4585 cpu_manager.go:414] "containerMap: removing container" podUID="96fb8d85-a90d-4c9f-9772-5da645f27721" containerName="revoke-keys"
Oct 30 15:58:27.830903 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.830827    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96fb8d85-a90d-4c9f-9772-5da645f27721" containerName="revoke-keys"
Oct 30 15:58:27.830903 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.830843    4585 cpu_manager.go:414] "containerMap: removing container" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy"
Oct 30 15:58:27.830903 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.830849    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy"
Oct 30 15:58:27.830903 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.830870    4585 cpu_manager.go:414] "containerMap: removing container" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy"
Oct 30 15:58:27.830903 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.830876    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35c54ae6-a861-4efc-8398-78b4610e0c40" containerName="istio-proxy"
Oct 30 15:58:27.831176 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.830982    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="96fb8d85-a90d-4c9f-9772-5da645f27721" containerName="revoke-keys"
Oct 30 15:58:27.831176 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.830993    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba4c3f49-4422-483f-b24f-41545ed46667" containerName="istio-proxy"
Oct 30 15:58:27.834862 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.834837    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.858834 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.858778    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"]
Oct 30 15:58:27.894634 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.894595    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.894822 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.894712    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.894822 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.894754    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.894939 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.894896    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkjdk\" (UniqueName: \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-kube-api-access-vkjdk\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.894939 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.894923    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.895060 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.895044    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f16429-e079-4aa7-adba-a8ceecd65b1f-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.895108 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.895092    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.895162 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.895120    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.895228 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.895212    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.996513 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.996474    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.996688 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.996527    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.996688 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.996584    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.996688 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.996613    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.996688 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.996675    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkjdk\" (UniqueName: \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-kube-api-access-vkjdk\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.996875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.996699    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.996875 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.996829    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f16429-e079-4aa7-adba-a8ceecd65b1f-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.996978 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.996873    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.996978 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.996901    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.997095 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.997073    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.997152 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.997091    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-certs\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.997152 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.997117    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-data\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.997299 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.997276    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-credential-socket\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.997662 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.997637    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f16429-e079-4aa7-adba-a8ceecd65b1f-istiod-ca-cert\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.999193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.999169    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-envoy\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:27.999403 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:27.999384    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-podinfo\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:28.021320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:28.021286    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-token\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:28.021477 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:28.021407    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkjdk\" (UniqueName: \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-kube-api-access-vkjdk\") pod \"e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") " pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:28.149797 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:28.149769    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:28.470165 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:28.470137    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"]
Oct 30 15:58:28.470281 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:58:28.470260    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69f16429_e079_4aa7_adba_a8ceecd65b1f.slice/crio-9cb3336880b99dbb0685f248ab552683701ea8f497a166d0cf014778e6bbb356 WatchSource:0}: Error finding container 9cb3336880b99dbb0685f248ab552683701ea8f497a166d0cf014778e6bbb356: Status 404 returned error can't find the container with id 9cb3336880b99dbb0685f248ab552683701ea8f497a166d0cf014778e6bbb356
Oct 30 15:58:28.472989 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:28.472957    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:58:28.473062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:28.473027    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:58:28.473062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:28.473054    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:58:28.505548 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:28.504971    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" event={"ID":"69f16429-e079-4aa7-adba-a8ceecd65b1f","Type":"ContainerStarted","Data":"9cb3336880b99dbb0685f248ab552683701ea8f497a166d0cf014778e6bbb356"}
Oct 30 15:58:29.512759 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:29.512721    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" event={"ID":"69f16429-e079-4aa7-adba-a8ceecd65b1f","Type":"ContainerStarted","Data":"928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd"}
Oct 30 15:58:29.597783 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:29.597722    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" podStartSLOduration=2.597703524 podStartE2EDuration="2.597703524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:58:29.597172978 +0000 UTC m=+1219.032777339" watchObservedRunningTime="2026-10-30 15:58:29.597703524 +0000 UTC m=+1219.033307873"
Oct 30 15:58:30.152376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.152149    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:30.152376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.152281    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 15:58:30.152376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.152324    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 15:58:30.421459 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.421340    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"]
Oct 30 15:58:30.445371 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.427112    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:30.460405 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.460337    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"]
Oct 30 15:58:30.474902 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.470702    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 15:58:30.474902 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.470764    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 15:58:30.474902 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.470812    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 15:58:30.474902 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.470859    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 15:58:30.656918 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.656885    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7597c6b3-qh9fp\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:30.657326 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.656936    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7597c6b3-qh9fp\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:30.657326 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.657044    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb2g5\" (UniqueName: \"kubernetes.io/projected/cc082cf8-74f3-4067-a26e-ceb24c6c88da-kube-api-access-bb2g5\") pod \"maas-api-revoke-sub-7597c6b3-qh9fp\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:30.758414 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.758294    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7597c6b3-qh9fp\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:30.758414 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.758386    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7597c6b3-qh9fp\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:30.758604 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.758443    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb2g5\" (UniqueName: \"kubernetes.io/projected/cc082cf8-74f3-4067-a26e-ceb24c6c88da-kube-api-access-bb2g5\") pod \"maas-api-revoke-sub-7597c6b3-qh9fp\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:30.759463 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.759432    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7597c6b3-qh9fp\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:30.761741 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.761722    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7597c6b3-qh9fp\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:30.782440 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.782403    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb2g5\" (UniqueName: \"kubernetes.io/projected/cc082cf8-74f3-4067-a26e-ceb24c6c88da-kube-api-access-bb2g5\") pod \"maas-api-revoke-sub-7597c6b3-qh9fp\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:30.805821 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:30.805795    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:31.158512 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:31.158468    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 15:58:31.158723 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:31.158527    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 15:58:31.159803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:31.159777    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"]
Oct 30 15:58:31.162384 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:58:31.162357    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc082cf8_74f3_4067_a26e_ceb24c6c88da.slice/crio-679753c62d544008af8d6b7a0dd17536b5b38df2a9e5e16c619321a900861e2c WatchSource:0}: Error finding container 679753c62d544008af8d6b7a0dd17536b5b38df2a9e5e16c619321a900861e2c: Status 404 returned error can't find the container with id 679753c62d544008af8d6b7a0dd17536b5b38df2a9e5e16c619321a900861e2c
Oct 30 15:58:31.574016 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:31.573972    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp" event={"ID":"cc082cf8-74f3-4067-a26e-ceb24c6c88da","Type":"ContainerStarted","Data":"c3c3cdf2361ceade769aa5026205a4116a9fd6344eab4907dbc3de01786dc095"}
Oct 30 15:58:31.574168 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:31.574049    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp" event={"ID":"cc082cf8-74f3-4067-a26e-ceb24c6c88da","Type":"ContainerStarted","Data":"679753c62d544008af8d6b7a0dd17536b5b38df2a9e5e16c619321a900861e2c"}
Oct 30 15:58:31.662447 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:31.662384    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp" podStartSLOduration=1.6623634109999998 podStartE2EDuration="1.662363411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:58:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:58:31.65969679 +0000 UTC m=+1221.095301158" watchObservedRunningTime="2026-10-30 15:58:31.662363411 +0000 UTC m=+1221.097967751"
Oct 30 15:58:32.152754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:32.152717    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 15:58:32.153063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:32.153035    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 15:58:32.583588 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:32.583492    4585 generic.go:356] "Generic (PLEG): container finished" podID="cc082cf8-74f3-4067-a26e-ceb24c6c88da" containerID="c3c3cdf2361ceade769aa5026205a4116a9fd6344eab4907dbc3de01786dc095" exitCode=0
Oct 30 15:58:32.583730 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:32.583578    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp" event={"ID":"cc082cf8-74f3-4067-a26e-ceb24c6c88da","Type":"ContainerDied","Data":"c3c3cdf2361ceade769aa5026205a4116a9fd6344eab4907dbc3de01786dc095"}
Oct 30 15:58:33.150802 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:33.150765    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 15:58:33.151462 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:33.151426    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 15:58:33.798520 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:33.798493    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:33.908812 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:33.908776    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-xks-ca\") pod \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") "
Oct 30 15:58:33.909007 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:33.908975    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc082cf8-74f3-4067-a26e-ceb24c6c88da-kube-api-access-bb2g5\" (UniqueName: \"kubernetes.io/projected/cc082cf8-74f3-4067-a26e-ceb24c6c88da-kube-api-access-bb2g5\") pod \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") "
Oct 30 15:58:33.909070 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:33.909048    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-openshift-ca\") pod \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\" (UID: \"cc082cf8-74f3-4067-a26e-ceb24c6c88da\") "
Oct 30 15:58:33.909887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:33.909855    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-openshift-ca" pod "cc082cf8-74f3-4067-a26e-ceb24c6c88da" (UID: "cc082cf8-74f3-4067-a26e-ceb24c6c88da"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:33.911230 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:33.911206    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-xks-ca" pod "cc082cf8-74f3-4067-a26e-ceb24c6c88da" (UID: "cc082cf8-74f3-4067-a26e-ceb24c6c88da"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:58:33.911327 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:33.911224    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc082cf8-74f3-4067-a26e-ceb24c6c88da-kube-api-access-bb2g5" pod "cc082cf8-74f3-4067-a26e-ceb24c6c88da" (UID: "cc082cf8-74f3-4067-a26e-ceb24c6c88da"). InnerVolumeSpecName "kube-api-access-bb2g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:34.011098 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:34.011055    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:34.011098 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:34.011098    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc082cf8-74f3-4067-a26e-ceb24c6c88da-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:34.011327 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:34.011111    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb2g5\" (UniqueName: \"kubernetes.io/projected/cc082cf8-74f3-4067-a26e-ceb24c6c88da-kube-api-access-bb2g5\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:34.153769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:34.153729    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 15:58:34.154419 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:34.154381    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 15:58:34.597035 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:34.597002    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"
Oct 30 15:58:34.597218 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:34.597003    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp" event={"ID":"cc082cf8-74f3-4067-a26e-ceb24c6c88da","Type":"ContainerDied","Data":"679753c62d544008af8d6b7a0dd17536b5b38df2a9e5e16c619321a900861e2c"}
Oct 30 15:58:34.597218 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:34.597143    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="679753c62d544008af8d6b7a0dd17536b5b38df2a9e5e16c619321a900861e2c"
Oct 30 15:58:35.150435 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:35.150391    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 15:58:35.150627 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:35.150471    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 15:58:35.481153 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:35.481058    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"]
Oct 30 15:58:35.481654 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:35.481262    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy" containerID="cri-o://baa79aec8b0c8b4d61718a6e8387f84bceb42f676adc01b68a7eb144b2a805f6" gracePeriod=30
Oct 30 15:58:35.486821 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:35.486791    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"]
Oct 30 15:58:35.487077 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:35.487050    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy" containerID="cri-o://87c56bfe4734c59c4bed9eaf4c096fd8a6aa869743b96fffe2fd46c7e67f4818" gracePeriod=30
Oct 30 15:58:35.507908 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:35.507870    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"]
Oct 30 15:58:35.508159 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:35.508131    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy" containerID="cri-o://3ce422b3b9516f9c8a957e595cd590864e6f3582467cce2c56fa21f661cc3999" gracePeriod=30
Oct 30 15:58:36.073638 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:36.073601    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 15:58:36.073966 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:36.073939    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 15:58:36.128059 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:36.128026    4585 patch_prober.go:29] interesting pod/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Oct 30 15:58:36.128232 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:36.128096    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 15:58:36.157611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:36.157578    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:36.157979 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:36.157955    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:36.158707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:36.158688    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:58:40.854626 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:40.854500    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:58:41.131277 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:41.127821    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:58:41.180776 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:41.179453    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:58:41.299374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:41.286434    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 15:58:41.327387 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:41.320435    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:58:42.130391 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:41.930788    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:58:42.140917 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.131082    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:58:42.150943 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.131726    4585 generic.go:356] "Generic (PLEG): container finished" podID="8b400747-55f6-49cb-a8dc-3b547323787e" containerID="baa79aec8b0c8b4d61718a6e8387f84bceb42f676adc01b68a7eb144b2a805f6" exitCode=0
Oct 30 15:58:42.182709 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.131755    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" event={"ID":"8b400747-55f6-49cb-a8dc-3b547323787e","Type":"ContainerDied","Data":"baa79aec8b0c8b4d61718a6e8387f84bceb42f676adc01b68a7eb144b2a805f6"}
Oct 30 15:58:42.183063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.183031    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:58:42.243383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.182433    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:58:42.243383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.183478    4585 scope.go:122] "RemoveContainer" containerID="732a4ec34374b4b62e5d300dba5c04827d45f2656296acf47a34d626f789c7fb"
Oct 30 15:58:42.363390 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.293494    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:58:42.364103 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.360796    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:58:42.568590 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.568528    4585 generic.go:356] "Generic (PLEG): container finished" podID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerID="3ce422b3b9516f9c8a957e595cd590864e6f3582467cce2c56fa21f661cc3999" exitCode=0
Oct 30 15:58:42.640365 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.640248    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:58:42.640365 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.640332    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" event={"ID":"91cc851d-7402-44c4-9f5c-88c5a5c6a90c","Type":"ContainerDied","Data":"3ce422b3b9516f9c8a957e595cd590864e6f3582467cce2c56fa21f661cc3999"}
Oct 30 15:58:42.671855 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.654387    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:58:42.671855 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.654462    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:58:42.671855 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.654497    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:58:42.913564 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.913436    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:58:42.916650 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.916626    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:58:42.943420 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.921541    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:58:42.959059 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.953502    4585 generic.go:356] "Generic (PLEG): container finished" podID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerID="87c56bfe4734c59c4bed9eaf4c096fd8a6aa869743b96fffe2fd46c7e67f4818" exitCode=0
Oct 30 15:58:42.959059 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.953595    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" event={"ID":"0ad89200-6a9a-4884-a94c-8056a9ebeb49","Type":"ContainerDied","Data":"87c56bfe4734c59c4bed9eaf4c096fd8a6aa869743b96fffe2fd46c7e67f4818"}
Oct 30 15:58:42.987385 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.987338    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:58:42.989154 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.989035    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:58:42.995365 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:42.995252    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:58:43.003116 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.000117    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:58:43.017376 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.013698    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044509    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-kube-api-access-m6ftm\" (UniqueName: \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-kube-api-access-m6ftm\") pod \"8b400747-55f6-49cb-a8dc-3b547323787e\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044569    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-credential-socket\") pod \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044599    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-socket\") pod \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044640    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-kube-api-access-vb4s7\" (UniqueName: \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-kube-api-access-vb4s7\") pod \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044676    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-envoy\") pod \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044711    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-data\") pod \"8b400747-55f6-49cb-a8dc-3b547323787e\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044747    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-data\") pod \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044774    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-certs\") pod \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044825    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-socket\") pod \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044855    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-socket\") pod \"8b400747-55f6-49cb-a8dc-3b547323787e\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044906    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istiod-ca-cert\") pod \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044940    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-envoy\") pod \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.044992    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b400747-55f6-49cb-a8dc-3b547323787e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b400747-55f6-49cb-a8dc-3b547323787e-istiod-ca-cert\") pod \"8b400747-55f6-49cb-a8dc-3b547323787e\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045047    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-data\") pod \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045082    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istiod-ca-cert\") pod \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045111    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-token\" (UniqueName: \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-token\") pod \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045157    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-podinfo\") pod \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\" (UID: \"91cc851d-7402-44c4-9f5c-88c5a5c6a90c\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045217    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-credential-socket\") pod \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045249    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-certs\") pod \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045282    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-podinfo\") pod \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045326    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-certs\") pod \"8b400747-55f6-49cb-a8dc-3b547323787e\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045405    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-token\" (UniqueName: \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-token\") pod \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045489    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-kube-api-access-64rxm\" (UniqueName: \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-kube-api-access-64rxm\") pod \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\" (UID: \"0ad89200-6a9a-4884-a94c-8056a9ebeb49\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045520    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-credential-socket\") pod \"8b400747-55f6-49cb-a8dc-3b547323787e\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045547    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-istio-token\" (UniqueName: \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-istio-token\") pod \"8b400747-55f6-49cb-a8dc-3b547323787e\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045584    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8b400747-55f6-49cb-a8dc-3b547323787e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b400747-55f6-49cb-a8dc-3b547323787e-istio-podinfo\") pod \"8b400747-55f6-49cb-a8dc-3b547323787e\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.045630    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-envoy\") pod \"8b400747-55f6-49cb-a8dc-3b547323787e\" (UID: \"8b400747-55f6-49cb-a8dc-3b547323787e\") "
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.047476    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b400747-55f6-49cb-a8dc-3b547323787e-istiod-ca-cert" pod "8b400747-55f6-49cb-a8dc-3b547323787e" (UID: "8b400747-55f6-49cb-a8dc-3b547323787e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.049103    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-socket" pod "8b400747-55f6-49cb-a8dc-3b547323787e" (UID: "8b400747-55f6-49cb-a8dc-3b547323787e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.049279    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-certs" pod "91cc851d-7402-44c4-9f5c-88c5a5c6a90c" (UID: "91cc851d-7402-44c4-9f5c-88c5a5c6a90c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.049479    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-kube-api-access-m6ftm" pod "8b400747-55f6-49cb-a8dc-3b547323787e" (UID: "8b400747-55f6-49cb-a8dc-3b547323787e"). InnerVolumeSpecName "kube-api-access-m6ftm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:43.049501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.049499    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-socket" pod "0ad89200-6a9a-4884-a94c-8056a9ebeb49" (UID: "0ad89200-6a9a-4884-a94c-8056a9ebeb49"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.053898 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.049687    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-data" pod "0ad89200-6a9a-4884-a94c-8056a9ebeb49" (UID: "0ad89200-6a9a-4884-a94c-8056a9ebeb49"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.053898 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.049843    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istiod-ca-cert" pod "91cc851d-7402-44c4-9f5c-88c5a5c6a90c" (UID: "91cc851d-7402-44c4-9f5c-88c5a5c6a90c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:43.053898 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.050447    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-credential-socket" pod "0ad89200-6a9a-4884-a94c-8056a9ebeb49" (UID: "0ad89200-6a9a-4884-a94c-8056a9ebeb49"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.053898 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.050940    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-data" pod "91cc851d-7402-44c4-9f5c-88c5a5c6a90c" (UID: "91cc851d-7402-44c4-9f5c-88c5a5c6a90c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.053898 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.051462    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-credential-socket" pod "8b400747-55f6-49cb-a8dc-3b547323787e" (UID: "8b400747-55f6-49cb-a8dc-3b547323787e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.053898 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.053717    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-envoy" pod "91cc851d-7402-44c4-9f5c-88c5a5c6a90c" (UID: "91cc851d-7402-44c4-9f5c-88c5a5c6a90c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.053898 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.053867    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-data" pod "8b400747-55f6-49cb-a8dc-3b547323787e" (UID: "8b400747-55f6-49cb-a8dc-3b547323787e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.054534 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.054507    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-certs" pod "0ad89200-6a9a-4884-a94c-8056a9ebeb49" (UID: "0ad89200-6a9a-4884-a94c-8056a9ebeb49"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.054893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.054864    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-token" pod "91cc851d-7402-44c4-9f5c-88c5a5c6a90c" (UID: "91cc851d-7402-44c4-9f5c-88c5a5c6a90c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:43.055236 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.055212    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istiod-ca-cert" pod "0ad89200-6a9a-4884-a94c-8056a9ebeb49" (UID: "0ad89200-6a9a-4884-a94c-8056a9ebeb49"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:43.055502 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.055482    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-certs" pod "8b400747-55f6-49cb-a8dc-3b547323787e" (UID: "8b400747-55f6-49cb-a8dc-3b547323787e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.055664 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.055636    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-credential-socket" pod "91cc851d-7402-44c4-9f5c-88c5a5c6a90c" (UID: "91cc851d-7402-44c4-9f5c-88c5a5c6a90c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.055842 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.055820    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-socket" pod "91cc851d-7402-44c4-9f5c-88c5a5c6a90c" (UID: "91cc851d-7402-44c4-9f5c-88c5a5c6a90c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.057038 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.056861    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-istio-token" pod "8b400747-55f6-49cb-a8dc-3b547323787e" (UID: "8b400747-55f6-49cb-a8dc-3b547323787e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:43.057785 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.057482    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b400747-55f6-49cb-a8dc-3b547323787e-istio-podinfo" pod "8b400747-55f6-49cb-a8dc-3b547323787e" (UID: "8b400747-55f6-49cb-a8dc-3b547323787e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:43.057863 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.057803    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-kube-api-access-64rxm" pod "0ad89200-6a9a-4884-a94c-8056a9ebeb49" (UID: "0ad89200-6a9a-4884-a94c-8056a9ebeb49"). InnerVolumeSpecName "kube-api-access-64rxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:43.058407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.058374    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-token" pod "0ad89200-6a9a-4884-a94c-8056a9ebeb49" (UID: "0ad89200-6a9a-4884-a94c-8056a9ebeb49"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:43.058555 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.058510    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-podinfo" pod "0ad89200-6a9a-4884-a94c-8056a9ebeb49" (UID: "0ad89200-6a9a-4884-a94c-8056a9ebeb49"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:43.058555 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.058540    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-kube-api-access-vb4s7" pod "91cc851d-7402-44c4-9f5c-88c5a5c6a90c" (UID: "91cc851d-7402-44c4-9f5c-88c5a5c6a90c"). InnerVolumeSpecName "kube-api-access-vb4s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:43.058692 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.058611    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-envoy" pod "8b400747-55f6-49cb-a8dc-3b547323787e" (UID: "8b400747-55f6-49cb-a8dc-3b547323787e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.060001 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.059972    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-podinfo" pod "91cc851d-7402-44c4-9f5c-88c5a5c6a90c" (UID: "91cc851d-7402-44c4-9f5c-88c5a5c6a90c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:58:43.060221 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.060197    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-envoy" pod "0ad89200-6a9a-4884-a94c-8056a9ebeb49" (UID: "0ad89200-6a9a-4884-a94c-8056a9ebeb49"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:58:43.147092 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147053    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147092 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147087    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147092 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147100    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147116    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147127    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147138    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147150    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64rxm\" (UniqueName: \"kubernetes.io/projected/0ad89200-6a9a-4884-a94c-8056a9ebeb49-kube-api-access-64rxm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147162    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147174    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147186    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b400747-55f6-49cb-a8dc-3b547323787e-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147198    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147213    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6ftm\" (UniqueName: \"kubernetes.io/projected/8b400747-55f6-49cb-a8dc-3b547323787e-kube-api-access-m6ftm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147228    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147243    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147258    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb4s7\" (UniqueName: \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-kube-api-access-vb4s7\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147271    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147284    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147294    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147306    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147319    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147331    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b400747-55f6-49cb-a8dc-3b547323787e-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147361    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147375    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147389    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b400747-55f6-49cb-a8dc-3b547323787e-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147401    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147414    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad89200-6a9a-4884-a94c-8056a9ebeb49-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.147659 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.147427    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91cc851d-7402-44c4-9f5c-88c5a5c6a90c-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:43.962307 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.962268    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"
Oct 30 15:58:43.962520 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.962271    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2" event={"ID":"8b400747-55f6-49cb-a8dc-3b547323787e","Type":"ContainerDied","Data":"dbf0f258d0d40df38df246977b0ad459e5411775dc100c184346e071ef79c329"}
Oct 30 15:58:43.962520 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.962436    4585 scope.go:122] "RemoveContainer" containerID="baa79aec8b0c8b4d61718a6e8387f84bceb42f676adc01b68a7eb144b2a805f6"
Oct 30 15:58:43.964221 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.964200    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"
Oct 30 15:58:43.964463 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.964197    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5" event={"ID":"91cc851d-7402-44c4-9f5c-88c5a5c6a90c","Type":"ContainerDied","Data":"3fc5ef7c7ac24235311c4d9d1141009e78e1bd0753052cf4b1167885d72a77ca"}
Oct 30 15:58:43.968031 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.968000    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv" event={"ID":"0ad89200-6a9a-4884-a94c-8056a9ebeb49","Type":"ContainerDied","Data":"9b596aeba57097e8b1b49aebfea9e660383f0706009de3045c7f64cef076bc99"}
Oct 30 15:58:43.969636 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.969613    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"
Oct 30 15:58:43.980426 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.980403    4585 scope.go:122] "RemoveContainer" containerID="3ce422b3b9516f9c8a957e595cd590864e6f3582467cce2c56fa21f661cc3999"
Oct 30 15:58:43.993341 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:43.993320    4585 scope.go:122] "RemoveContainer" containerID="87c56bfe4734c59c4bed9eaf4c096fd8a6aa869743b96fffe2fd46c7e67f4818"
Oct 30 15:58:44.057166 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.057129    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"]
Oct 30 15:58:44.067805 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.067768    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-s9lv2"]
Oct 30 15:58:44.102175 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.102141    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"]
Oct 30 15:58:44.116535 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.116498    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-cpzpv"]
Oct 30 15:58:44.151155 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.151122    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"]
Oct 30 15:58:44.154908 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.154874    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b24e7b-openshift-default-b6dd9956f-r9sd5"]
Oct 30 15:58:44.248914 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.248881    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"]
Oct 30 15:58:44.249442 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249423    4585 cpu_manager.go:414] "containerMap: removing container" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy"
Oct 30 15:58:44.249442 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249441    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249456    4585 cpu_manager.go:414] "containerMap: removing container" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249462    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249473    4585 cpu_manager.go:414] "containerMap: removing container" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249481    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249500    4585 cpu_manager.go:414] "containerMap: removing container" podUID="cc082cf8-74f3-4067-a26e-ceb24c6c88da" containerName="revoke-keys"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249505    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc082cf8-74f3-4067-a26e-ceb24c6c88da" containerName="revoke-keys"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249512    4585 cpu_manager.go:414] "containerMap: removing container" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249516    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249592    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc082cf8-74f3-4067-a26e-ceb24c6c88da" containerName="revoke-keys"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249600    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy"
Oct 30 15:58:44.249602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249609    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" containerName="istio-proxy"
Oct 30 15:58:44.250151 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249616    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" containerName="istio-proxy"
Oct 30 15:58:44.250151 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.249624    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" containerName="istio-proxy"
Oct 30 15:58:44.254771 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.254742    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.272081 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.272049    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"]
Oct 30 15:58:44.272490 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.272468    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 15:58:44.272938 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.272919    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 15:58:44.280368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.280317    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 15:58:44.280368 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.280326    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 15:58:44.362644 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.362609    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wb5z\" (UniqueName: \"kubernetes.io/projected/90c831a0-898d-479e-8c5e-c7170b97bef7-kube-api-access-7wb5z\") pod \"maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.362961 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.362734    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.363040 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.362972    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.464107 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.464003    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.464107 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.464103    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wb5z\" (UniqueName: \"kubernetes.io/projected/90c831a0-898d-479e-8c5e-c7170b97bef7-kube-api-access-7wb5z\") pod \"maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.464365 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.464152    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.464769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.464741    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.466802 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.466776    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.487578 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.487546    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wb5z\" (UniqueName: \"kubernetes.io/projected/90c831a0-898d-479e-8c5e-c7170b97bef7-kube-api-access-7wb5z\") pod \"maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.573107 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.573069    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:44.703694 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.703661    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ad89200-6a9a-4884-a94c-8056a9ebeb49" path="/var/lib/kubelet/pods/0ad89200-6a9a-4884-a94c-8056a9ebeb49/volumes"
Oct 30 15:58:44.704139 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.704124    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b400747-55f6-49cb-a8dc-3b547323787e" path="/var/lib/kubelet/pods/8b400747-55f6-49cb-a8dc-3b547323787e/volumes"
Oct 30 15:58:44.704573 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.704559    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91cc851d-7402-44c4-9f5c-88c5a5c6a90c" path="/var/lib/kubelet/pods/91cc851d-7402-44c4-9f5c-88c5a5c6a90c/volumes"
Oct 30 15:58:44.876383 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.876337    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"]
Oct 30 15:58:44.877510 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:58:44.877486    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c831a0_898d_479e_8c5e_c7170b97bef7.slice/crio-ae2f7e1da223080ea4c11339d33a1e1ac25a78db842df698579a3b584ba4613f WatchSource:0}: Error finding container ae2f7e1da223080ea4c11339d33a1e1ac25a78db842df698579a3b584ba4613f: Status 404 returned error can't find the container with id ae2f7e1da223080ea4c11339d33a1e1ac25a78db842df698579a3b584ba4613f
Oct 30 15:58:44.976685 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:44.976649    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm" event={"ID":"90c831a0-898d-479e-8c5e-c7170b97bef7","Type":"ContainerStarted","Data":"ae2f7e1da223080ea4c11339d33a1e1ac25a78db842df698579a3b584ba4613f"}
Oct 30 15:58:45.989722 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:45.989625    4585 generic.go:356] "Generic (PLEG): container finished" podID="90c831a0-898d-479e-8c5e-c7170b97bef7" containerID="65ea9172d926eab59e88c9af2d596d6ac994d992d9c21935e1778df523ceef91" exitCode=0
Oct 30 15:58:45.990111 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:45.989724    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm" event={"ID":"90c831a0-898d-479e-8c5e-c7170b97bef7","Type":"ContainerDied","Data":"65ea9172d926eab59e88c9af2d596d6ac994d992d9c21935e1778df523ceef91"}
Oct 30 15:58:46.559445 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:46.559405    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"]
Oct 30 15:58:47.143503 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:47.143477    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:47.200746 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:47.200710    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90c831a0-898d-479e-8c5e-c7170b97bef7-kube-api-access-7wb5z\" (UniqueName: \"kubernetes.io/projected/90c831a0-898d-479e-8c5e-c7170b97bef7-kube-api-access-7wb5z\") pod \"90c831a0-898d-479e-8c5e-c7170b97bef7\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") "
Oct 30 15:58:47.200929 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:47.200872    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-xks-ca\") pod \"90c831a0-898d-479e-8c5e-c7170b97bef7\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") "
Oct 30 15:58:47.200988 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:47.200949    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-openshift-ca\") pod \"90c831a0-898d-479e-8c5e-c7170b97bef7\" (UID: \"90c831a0-898d-479e-8c5e-c7170b97bef7\") "
Oct 30 15:58:47.201418 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:47.201385    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-openshift-ca" pod "90c831a0-898d-479e-8c5e-c7170b97bef7" (UID: "90c831a0-898d-479e-8c5e-c7170b97bef7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:58:47.201542 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:47.201501    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:47.202959 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:47.202935    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-xks-ca" pod "90c831a0-898d-479e-8c5e-c7170b97bef7" (UID: "90c831a0-898d-479e-8c5e-c7170b97bef7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:58:47.203151 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:47.203135    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90c831a0-898d-479e-8c5e-c7170b97bef7-kube-api-access-7wb5z" pod "90c831a0-898d-479e-8c5e-c7170b97bef7" (UID: "90c831a0-898d-479e-8c5e-c7170b97bef7"). InnerVolumeSpecName "kube-api-access-7wb5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:58:47.303259 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:47.303174    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wb5z\" (UniqueName: \"kubernetes.io/projected/90c831a0-898d-479e-8c5e-c7170b97bef7-kube-api-access-7wb5z\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:47.303259 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:47.303207    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/90c831a0-898d-479e-8c5e-c7170b97bef7-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:58:47.971522 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:58:47.971481    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:58:48.008454 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:48.008416    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm" event={"ID":"90c831a0-898d-479e-8c5e-c7170b97bef7","Type":"ContainerDied","Data":"ae2f7e1da223080ea4c11339d33a1e1ac25a78db842df698579a3b584ba4613f"}
Oct 30 15:58:48.008664 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:48.008567    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae2f7e1da223080ea4c11339d33a1e1ac25a78db842df698579a3b584ba4613f"
Oct 30 15:58:48.008664 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:48.008658    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"
Oct 30 15:58:49.462877 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:49.462812    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"]
Oct 30 15:58:54.731482 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:54.731446    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"]
Oct 30 15:58:55.436303 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:55.436270    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"]
Oct 30 15:58:56.886063 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.886028    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"]
Oct 30 15:58:56.886527 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.886513    4585 cpu_manager.go:414] "containerMap: removing container" podUID="90c831a0-898d-479e-8c5e-c7170b97bef7" containerName="revoke-keys"
Oct 30 15:58:56.886572 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.886528    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90c831a0-898d-479e-8c5e-c7170b97bef7" containerName="revoke-keys"
Oct 30 15:58:56.886626 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.886617    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="90c831a0-898d-479e-8c5e-c7170b97bef7" containerName="revoke-keys"
Oct 30 15:58:56.899002 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.897390    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:56.910696 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.910646    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"]
Oct 30 15:58:56.940102 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.940059    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-71729861-4mzwc\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:56.940324 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.940185    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-71729861-4mzwc\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:56.940424 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.940334    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc5gw\" (UniqueName: \"kubernetes.io/projected/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-kube-api-access-qc5gw\") pod \"maas-api-revoke-sub-71729861-4mzwc\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:56.984134 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.984099    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"]
Oct 30 15:58:56.988248 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.988221    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:56.996798 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.996767    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 15:58:56.997305 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.997229    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 15:58:56.998000 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.997517    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 15:58:56.998236 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:56.998191    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 15:58:57.011453 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.011421    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"]
Oct 30 15:58:57.041470 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.041417    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a5343dac-87pgc\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:57.041680 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.041523    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-71729861-4mzwc\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:57.041680 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.041616    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-71729861-4mzwc\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:57.041789 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.041726    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc5gw\" (UniqueName: \"kubernetes.io/projected/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-kube-api-access-qc5gw\") pod \"maas-api-revoke-sub-71729861-4mzwc\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:57.041789 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.041775    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a5343dac-87pgc\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:57.041887 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.041830    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4678\" (UniqueName: \"kubernetes.io/projected/d5ca140b-084b-4889-9881-f28f7dd09b56-kube-api-access-d4678\") pod \"maas-api-revoke-sub-a5343dac-87pgc\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:57.042418 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.042390    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-71729861-4mzwc\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:57.047396 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.045040    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-71729861-4mzwc\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:57.067054 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.067022    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc5gw\" (UniqueName: \"kubernetes.io/projected/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-kube-api-access-qc5gw\") pod \"maas-api-revoke-sub-71729861-4mzwc\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:57.077816 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.077786    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"]
Oct 30 15:58:57.082194 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.082169    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.098567 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.098535    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"]
Oct 30 15:58:57.143049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.142946    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgd8x\" (UniqueName: \"kubernetes.io/projected/41f752fd-9135-461f-a763-a5fdb474161f-kube-api-access-hgd8x\") pod \"maas-api-revoke-sub-1e54e2f2-6pnfb\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.143049 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.143024    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a5343dac-87pgc\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:57.143282 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.143106    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4678\" (UniqueName: \"kubernetes.io/projected/d5ca140b-084b-4889-9881-f28f7dd09b56-kube-api-access-d4678\") pod \"maas-api-revoke-sub-a5343dac-87pgc\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:57.143282 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.143230    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a5343dac-87pgc\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:57.143282 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.143267    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1e54e2f2-6pnfb\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.143472 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.143407    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1e54e2f2-6pnfb\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.143759 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.143738    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a5343dac-87pgc\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:57.145609 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.145583    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a5343dac-87pgc\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:57.168770 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.168739    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4678\" (UniqueName: \"kubernetes.io/projected/d5ca140b-084b-4889-9881-f28f7dd09b56-kube-api-access-d4678\") pod \"maas-api-revoke-sub-a5343dac-87pgc\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:57.219270 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.219235    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:58:57.244455 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.244415    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgd8x\" (UniqueName: \"kubernetes.io/projected/41f752fd-9135-461f-a763-a5fdb474161f-kube-api-access-hgd8x\") pod \"maas-api-revoke-sub-1e54e2f2-6pnfb\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.244612 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.244596    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1e54e2f2-6pnfb\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.244678 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.244653    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1e54e2f2-6pnfb\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.245506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.245474    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1e54e2f2-6pnfb\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.247369 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.247331    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1e54e2f2-6pnfb\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.273091 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.273059    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgd8x\" (UniqueName: \"kubernetes.io/projected/41f752fd-9135-461f-a763-a5fdb474161f-kube-api-access-hgd8x\") pod \"maas-api-revoke-sub-1e54e2f2-6pnfb\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.309021 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.308989    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:58:57.409880 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.399573    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:58:57.527799 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:58:57.527766    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcdb205_4db0_4ef8_a2d8_b8b9e5597ed2.slice/crio-91b79c8e7f782dff7a92bbd5053b1ac226cdb2fef2dd348c766f39b880268c44 WatchSource:0}: Error finding container 91b79c8e7f782dff7a92bbd5053b1ac226cdb2fef2dd348c766f39b880268c44: Status 404 returned error can't find the container with id 91b79c8e7f782dff7a92bbd5053b1ac226cdb2fef2dd348c766f39b880268c44
Oct 30 15:58:57.532071 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.532042    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"]
Oct 30 15:58:57.660405 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.660265    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"]
Oct 30 15:58:57.671045 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:58:57.671013    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ca140b_084b_4889_9881_f28f7dd09b56.slice/crio-d153f4cacd1c45641e78ec54d7f7d8e57fef71c6a91cd9e2ec84d275bb5471d0 WatchSource:0}: Error finding container d153f4cacd1c45641e78ec54d7f7d8e57fef71c6a91cd9e2ec84d275bb5471d0: Status 404 returned error can't find the container with id d153f4cacd1c45641e78ec54d7f7d8e57fef71c6a91cd9e2ec84d275bb5471d0
Oct 30 15:58:57.732011 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:58:57.731974    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f752fd_9135_461f_a763_a5fdb474161f.slice/crio-108a337300a7172e1afa95e6943179a7aa9052fb8aaf561f18fce6ac43d06e3b WatchSource:0}: Error finding container 108a337300a7172e1afa95e6943179a7aa9052fb8aaf561f18fce6ac43d06e3b: Status 404 returned error can't find the container with id 108a337300a7172e1afa95e6943179a7aa9052fb8aaf561f18fce6ac43d06e3b
Oct 30 15:58:57.732832 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:57.732804    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"]
Oct 30 15:58:58.082319 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:58.082280    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc" event={"ID":"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2","Type":"ContainerStarted","Data":"2eedf1beaf6b440398b6d324ba64d36045ea7fdf84017be0948c920cb89b2539"}
Oct 30 15:58:58.082800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:58.082375    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc" event={"ID":"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2","Type":"ContainerStarted","Data":"91b79c8e7f782dff7a92bbd5053b1ac226cdb2fef2dd348c766f39b880268c44"}
Oct 30 15:58:58.084052 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:58.084019    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc" event={"ID":"d5ca140b-084b-4889-9881-f28f7dd09b56","Type":"ContainerStarted","Data":"fb82fd4f5e5a3fad62b9968dfdbefa2e67671ca994866c939c9bd09586e42a63"}
Oct 30 15:58:58.084171 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:58.084070    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc" event={"ID":"d5ca140b-084b-4889-9881-f28f7dd09b56","Type":"ContainerStarted","Data":"d153f4cacd1c45641e78ec54d7f7d8e57fef71c6a91cd9e2ec84d275bb5471d0"}
Oct 30 15:58:58.085493 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:58.085456    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb" event={"ID":"41f752fd-9135-461f-a763-a5fdb474161f","Type":"ContainerStarted","Data":"2c593d27b6c4d70e35c557189de58d6661aa58d7a8fc965fdf8eb80394a1f03f"}
Oct 30 15:58:58.085594 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:58.085511    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb" event={"ID":"41f752fd-9135-461f-a763-a5fdb474161f","Type":"ContainerStarted","Data":"108a337300a7172e1afa95e6943179a7aa9052fb8aaf561f18fce6ac43d06e3b"}
Oct 30 15:58:58.123803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:58.123751    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc" podStartSLOduration=2.123737598 podStartE2EDuration="2.123737598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:58:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:58:58.123041712 +0000 UTC m=+1247.558646060" watchObservedRunningTime="2026-10-30 15:58:58.123737598 +0000 UTC m=+1247.559341946"
Oct 30 15:58:58.163065 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:58.163002    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc" podStartSLOduration=2.1629841 podStartE2EDuration="2.1629841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:58:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:58:58.162180032 +0000 UTC m=+1247.597784380" watchObservedRunningTime="2026-10-30 15:58:58.1629841 +0000 UTC m=+1247.598588452"
Oct 30 15:58:58.202160 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:58.202061    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb" podStartSLOduration=1.202048523 podStartE2EDuration="1.202048523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:58:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:58:58.20142508 +0000 UTC m=+1247.637029428" watchObservedRunningTime="2026-10-30 15:58:58.202048523 +0000 UTC m=+1247.637652945"
Oct 30 15:58:59.093031 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:59.092999    4585 generic.go:356] "Generic (PLEG): container finished" podID="d5ca140b-084b-4889-9881-f28f7dd09b56" containerID="fb82fd4f5e5a3fad62b9968dfdbefa2e67671ca994866c939c9bd09586e42a63" exitCode=0
Oct 30 15:58:59.093465 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:59.093085    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc" event={"ID":"d5ca140b-084b-4889-9881-f28f7dd09b56","Type":"ContainerDied","Data":"fb82fd4f5e5a3fad62b9968dfdbefa2e67671ca994866c939c9bd09586e42a63"}
Oct 30 15:58:59.094675 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:59.094650    4585 generic.go:356] "Generic (PLEG): container finished" podID="41f752fd-9135-461f-a763-a5fdb474161f" containerID="2c593d27b6c4d70e35c557189de58d6661aa58d7a8fc965fdf8eb80394a1f03f" exitCode=0
Oct 30 15:58:59.094812 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:59.094723    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb" event={"ID":"41f752fd-9135-461f-a763-a5fdb474161f","Type":"ContainerDied","Data":"2c593d27b6c4d70e35c557189de58d6661aa58d7a8fc965fdf8eb80394a1f03f"}
Oct 30 15:58:59.096088 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:59.096042    4585 generic.go:356] "Generic (PLEG): container finished" podID="5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2" containerID="2eedf1beaf6b440398b6d324ba64d36045ea7fdf84017be0948c920cb89b2539" exitCode=0
Oct 30 15:58:59.096208 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:59.096127    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc" event={"ID":"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2","Type":"ContainerDied","Data":"2eedf1beaf6b440398b6d324ba64d36045ea7fdf84017be0948c920cb89b2539"}
Oct 30 15:58:59.732806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:58:59.732728    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg" podUID="2598a7ca-70ff-4bd2-ae89-fa742412fa98" containerName="payload-pre-processing" containerID="cri-o://239bb281af2a6ddefa6abe67a56aa4ed1c5d508068bee9d2a3886ff13ad0029e" gracePeriod=25
Oct 30 15:59:00.310231 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.310197    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:59:00.383930 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.383900    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:59:00.387983 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.387959    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:59:00.398830 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.398801    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-openshift-ca\") pod \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") "
Oct 30 15:59:00.398991 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.398873    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-kube-api-access-qc5gw\" (UniqueName: \"kubernetes.io/projected/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-kube-api-access-qc5gw\") pod \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") "
Oct 30 15:59:00.398991 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.398931    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-xks-ca\") pod \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\" (UID: \"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2\") "
Oct 30 15:59:00.399674 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.399645    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-openshift-ca" pod "5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2" (UID: "5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:00.401156 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.401129    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-kube-api-access-qc5gw" pod "5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2" (UID: "5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2"). InnerVolumeSpecName "kube-api-access-qc5gw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:00.401647 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.401622    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-xks-ca" pod "5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2" (UID: "5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:59:00.437152 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.437099    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4" podUID="547f6ffe-6f5a-4dac-8521-4e92df79f82e" containerName="payload-pre-processing" containerID="cri-o://92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f" gracePeriod=25
Oct 30 15:59:00.500886 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.500850    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-xks-ca\") pod \"41f752fd-9135-461f-a763-a5fdb474161f\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") "
Oct 30 15:59:00.501077 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.500907    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5ca140b-084b-4889-9881-f28f7dd09b56-kube-api-access-d4678\" (UniqueName: \"kubernetes.io/projected/d5ca140b-084b-4889-9881-f28f7dd09b56-kube-api-access-d4678\") pod \"d5ca140b-084b-4889-9881-f28f7dd09b56\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") "
Oct 30 15:59:00.501077 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.501036    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-openshift-ca\") pod \"d5ca140b-084b-4889-9881-f28f7dd09b56\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") "
Oct 30 15:59:00.501200 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.501112    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41f752fd-9135-461f-a763-a5fdb474161f-kube-api-access-hgd8x\" (UniqueName: \"kubernetes.io/projected/41f752fd-9135-461f-a763-a5fdb474161f-kube-api-access-hgd8x\") pod \"41f752fd-9135-461f-a763-a5fdb474161f\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") "
Oct 30 15:59:00.501200 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.501145    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-xks-ca\") pod \"d5ca140b-084b-4889-9881-f28f7dd09b56\" (UID: \"d5ca140b-084b-4889-9881-f28f7dd09b56\") "
Oct 30 15:59:00.501200 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.501175    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-openshift-ca\") pod \"41f752fd-9135-461f-a763-a5fdb474161f\" (UID: \"41f752fd-9135-461f-a763-a5fdb474161f\") "
Oct 30 15:59:00.501678 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.501438    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-openshift-ca" pod "d5ca140b-084b-4889-9881-f28f7dd09b56" (UID: "d5ca140b-084b-4889-9881-f28f7dd09b56"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:00.501800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.501722    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:00.501800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.501743    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qc5gw\" (UniqueName: \"kubernetes.io/projected/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-kube-api-access-qc5gw\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:00.501800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.501757    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:00.501800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.501772    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:00.502212 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.502180    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-openshift-ca" pod "41f752fd-9135-461f-a763-a5fdb474161f" (UID: "41f752fd-9135-461f-a763-a5fdb474161f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:00.503209 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.503187    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-xks-ca" pod "d5ca140b-084b-4889-9881-f28f7dd09b56" (UID: "d5ca140b-084b-4889-9881-f28f7dd09b56"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:59:00.503293 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.503249    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5ca140b-084b-4889-9881-f28f7dd09b56-kube-api-access-d4678" pod "d5ca140b-084b-4889-9881-f28f7dd09b56" (UID: "d5ca140b-084b-4889-9881-f28f7dd09b56"). InnerVolumeSpecName "kube-api-access-d4678". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:00.503293 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.503248    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-xks-ca" pod "41f752fd-9135-461f-a763-a5fdb474161f" (UID: "41f752fd-9135-461f-a763-a5fdb474161f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:59:00.503697 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.503682    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41f752fd-9135-461f-a763-a5fdb474161f-kube-api-access-hgd8x" pod "41f752fd-9135-461f-a763-a5fdb474161f" (UID: "41f752fd-9135-461f-a763-a5fdb474161f"). InnerVolumeSpecName "kube-api-access-hgd8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:00.603502 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.603471    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4678\" (UniqueName: \"kubernetes.io/projected/d5ca140b-084b-4889-9881-f28f7dd09b56-kube-api-access-d4678\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:00.603502 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.603500    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgd8x\" (UniqueName: \"kubernetes.io/projected/41f752fd-9135-461f-a763-a5fdb474161f-kube-api-access-hgd8x\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:00.603502 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.603513    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d5ca140b-084b-4889-9881-f28f7dd09b56-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:00.603757 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.603522    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:00.603757 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:00.603531    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/41f752fd-9135-461f-a763-a5fdb474161f-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:01.108545 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.108459    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"
Oct 30 15:59:01.108545 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.108490    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc" event={"ID":"d5ca140b-084b-4889-9881-f28f7dd09b56","Type":"ContainerDied","Data":"d153f4cacd1c45641e78ec54d7f7d8e57fef71c6a91cd9e2ec84d275bb5471d0"}
Oct 30 15:59:01.108545 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.108540    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d153f4cacd1c45641e78ec54d7f7d8e57fef71c6a91cd9e2ec84d275bb5471d0"
Oct 30 15:59:01.110150 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.110132    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"
Oct 30 15:59:01.110150 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.110141    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb" event={"ID":"41f752fd-9135-461f-a763-a5fdb474161f","Type":"ContainerDied","Data":"108a337300a7172e1afa95e6943179a7aa9052fb8aaf561f18fce6ac43d06e3b"}
Oct 30 15:59:01.110326 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.110181    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="108a337300a7172e1afa95e6943179a7aa9052fb8aaf561f18fce6ac43d06e3b"
Oct 30 15:59:01.111751 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.111733    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc" event={"ID":"5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2","Type":"ContainerDied","Data":"91b79c8e7f782dff7a92bbd5053b1ac226cdb2fef2dd348c766f39b880268c44"}
Oct 30 15:59:01.111863 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.111757    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"
Oct 30 15:59:01.111863 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.111766    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91b79c8e7f782dff7a92bbd5053b1ac226cdb2fef2dd348c766f39b880268c44"
Oct 30 15:59:01.579765 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.579730    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main" containerID="cri-o://d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608" gracePeriod=45
Oct 30 15:59:01.858372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:01.858334    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:59:02.019180 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.019145    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-dshm\") pod \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") "
Oct 30 15:59:02.019361 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.019203    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tmp-dir\") pod \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") "
Oct 30 15:59:02.019361 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.019238    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-model-cache\") pod \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") "
Oct 30 15:59:02.019516 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.019490    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-model-cache" pod "1ae2931d-aa72-4b77-bd29-78d29060ff8f" (UID: "1ae2931d-aa72-4b77-bd29-78d29060ff8f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:02.019582 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.019501    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tls-certs\") pod \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") "
Oct 30 15:59:02.019582 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.019500    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tmp-dir" pod "1ae2931d-aa72-4b77-bd29-78d29060ff8f" (UID: "1ae2931d-aa72-4b77-bd29-78d29060ff8f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:02.019582 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.019554    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ae2931d-aa72-4b77-bd29-78d29060ff8f-kube-api-access-2lcp9\" (UniqueName: \"kubernetes.io/projected/1ae2931d-aa72-4b77-bd29-78d29060ff8f-kube-api-access-2lcp9\") pod \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") "
Oct 30 15:59:02.019738 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.019605    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-home\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-home\") pod \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\" (UID: \"1ae2931d-aa72-4b77-bd29-78d29060ff8f\") "
Oct 30 15:59:02.019904 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.019882    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-home" pod "1ae2931d-aa72-4b77-bd29-78d29060ff8f" (UID: "1ae2931d-aa72-4b77-bd29-78d29060ff8f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:02.020163 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.020143    4585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-home\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:02.020251 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.020165    4585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tmp-dir\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:02.020251 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.020180    4585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-model-cache\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:02.021522 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.021497    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-dshm" pod "1ae2931d-aa72-4b77-bd29-78d29060ff8f" (UID: "1ae2931d-aa72-4b77-bd29-78d29060ff8f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:02.021627 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.021614    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ae2931d-aa72-4b77-bd29-78d29060ff8f-kube-api-access-2lcp9" pod "1ae2931d-aa72-4b77-bd29-78d29060ff8f" (UID: "1ae2931d-aa72-4b77-bd29-78d29060ff8f"). InnerVolumeSpecName "kube-api-access-2lcp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:02.021701 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.021683    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tls-certs" pod "1ae2931d-aa72-4b77-bd29-78d29060ff8f" (UID: "1ae2931d-aa72-4b77-bd29-78d29060ff8f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:59:02.119123 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.119032    4585 generic.go:356] "Generic (PLEG): container finished" podID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerID="d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608" exitCode=0
Oct 30 15:59:02.119123 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.119079    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"
Oct 30 15:59:02.119377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.119120    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" event={"ID":"1ae2931d-aa72-4b77-bd29-78d29060ff8f","Type":"ContainerDied","Data":"d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608"}
Oct 30 15:59:02.119377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.119171    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv" event={"ID":"1ae2931d-aa72-4b77-bd29-78d29060ff8f","Type":"ContainerDied","Data":"f382de9295ec9995fd761507b6c023d87ddaeafc337ee0e05bb067e09f319f5e"}
Oct 30 15:59:02.119377 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.119193    4585 scope.go:122] "RemoveContainer" containerID="d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608"
Oct 30 15:59:02.121710 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.121683    4585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ae2931d-aa72-4b77-bd29-78d29060ff8f-tls-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:02.121805 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.121713    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lcp9\" (UniqueName: \"kubernetes.io/projected/1ae2931d-aa72-4b77-bd29-78d29060ff8f-kube-api-access-2lcp9\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:02.121805 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.121727    4585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ae2931d-aa72-4b77-bd29-78d29060ff8f-dshm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:02.131077 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.131055    4585 scope.go:122] "RemoveContainer" containerID="d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608"
Oct 30 15:59:02.131572 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:59:02.131554    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608\": container with ID starting with d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608 not found: ID does not exist" containerID="d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608"
Oct 30 15:59:02.131636 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.131581    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608"} err="failed to get container status \"d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608\": rpc error: code = NotFound desc = could not find container \"d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608\": container with ID starting with d99151bef6a437b8fa37490546e52f99b2d54a18e5b4fadc0584a848d464e608 not found: ID does not exist"
Oct 30 15:59:02.177648 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.177613    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"]
Oct 30 15:59:02.189520 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.189488    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-040f9a/simulator-w3-040f9a-kserve-69d8bc4dd6-mhspv"]
Oct 30 15:59:02.702594 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:02.702559    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" path="/var/lib/kubelet/pods/1ae2931d-aa72-4b77-bd29-78d29060ff8f/volumes"
Oct 30 15:59:04.484392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.484332    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main" containerID="cri-o://1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae" gracePeriod=45
Oct 30 15:59:04.756874 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.756269    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:59:04.857613 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.857574    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-model-cache\") pod \"44baf777-0b73-472d-8c5f-cc7b693f0409\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") "
Oct 30 15:59:04.857812 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.857635    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-dshm\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-dshm\") pod \"44baf777-0b73-472d-8c5f-cc7b693f0409\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") "
Oct 30 15:59:04.857812 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.857691    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-tmp-dir\") pod \"44baf777-0b73-472d-8c5f-cc7b693f0409\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") "
Oct 30 15:59:04.857812 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.857724    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44baf777-0b73-472d-8c5f-cc7b693f0409-kube-api-access-nd55b\" (UniqueName: \"kubernetes.io/projected/44baf777-0b73-472d-8c5f-cc7b693f0409-kube-api-access-nd55b\") pod \"44baf777-0b73-472d-8c5f-cc7b693f0409\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") "
Oct 30 15:59:04.857812 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.857778    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44baf777-0b73-472d-8c5f-cc7b693f0409-tls-certs\" (UniqueName: \"kubernetes.io/secret/44baf777-0b73-472d-8c5f-cc7b693f0409-tls-certs\") pod \"44baf777-0b73-472d-8c5f-cc7b693f0409\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") "
Oct 30 15:59:04.857812 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.857811    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-home\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-home\") pod \"44baf777-0b73-472d-8c5f-cc7b693f0409\" (UID: \"44baf777-0b73-472d-8c5f-cc7b693f0409\") "
Oct 30 15:59:04.858654 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.858626    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-home" pod "44baf777-0b73-472d-8c5f-cc7b693f0409" (UID: "44baf777-0b73-472d-8c5f-cc7b693f0409"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:04.858883 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.858853    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-tmp-dir" pod "44baf777-0b73-472d-8c5f-cc7b693f0409" (UID: "44baf777-0b73-472d-8c5f-cc7b693f0409"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:04.858953 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.858937    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-model-cache" pod "44baf777-0b73-472d-8c5f-cc7b693f0409" (UID: "44baf777-0b73-472d-8c5f-cc7b693f0409"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:04.861340 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.861259    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-dshm" pod "44baf777-0b73-472d-8c5f-cc7b693f0409" (UID: "44baf777-0b73-472d-8c5f-cc7b693f0409"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:04.861340 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.861301    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44baf777-0b73-472d-8c5f-cc7b693f0409-kube-api-access-nd55b" pod "44baf777-0b73-472d-8c5f-cc7b693f0409" (UID: "44baf777-0b73-472d-8c5f-cc7b693f0409"). InnerVolumeSpecName "kube-api-access-nd55b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:04.864770 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.864743    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44baf777-0b73-472d-8c5f-cc7b693f0409-tls-certs" pod "44baf777-0b73-472d-8c5f-cc7b693f0409" (UID: "44baf777-0b73-472d-8c5f-cc7b693f0409"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:59:04.959417 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.959373    4585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-model-cache\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:04.959417 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.959415    4585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-dshm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:04.959417 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.959423    4585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-tmp-dir\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:04.959417 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.959432    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd55b\" (UniqueName: \"kubernetes.io/projected/44baf777-0b73-472d-8c5f-cc7b693f0409-kube-api-access-nd55b\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:04.959698 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.959441    4585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44baf777-0b73-472d-8c5f-cc7b693f0409-tls-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:04.959698 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:04.959449    4585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44baf777-0b73-472d-8c5f-cc7b693f0409-home\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:05.141058 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.140968    4585 generic.go:356] "Generic (PLEG): container finished" podID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerID="1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae" exitCode=0
Oct 30 15:59:05.141223 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.141134    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" event={"ID":"44baf777-0b73-472d-8c5f-cc7b693f0409","Type":"ContainerDied","Data":"1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae"}
Oct 30 15:59:05.141223 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.141178    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg" event={"ID":"44baf777-0b73-472d-8c5f-cc7b693f0409","Type":"ContainerDied","Data":"fad5a8e12f208524fc023d454b0568514faad13c7451f8d5b585254a8769429c"}
Oct 30 15:59:05.141223 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.141203    4585 scope.go:122] "RemoveContainer" containerID="1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae"
Oct 30 15:59:05.141412 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.141364    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"
Oct 30 15:59:05.158114 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.157946    4585 scope.go:122] "RemoveContainer" containerID="1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae"
Oct 30 15:59:05.158320 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:59:05.158280    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae\": container with ID starting with 1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae not found: ID does not exist" containerID="1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae"
Oct 30 15:59:05.158427 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.158330    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae"} err="failed to get container status \"1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae\": rpc error: code = NotFound desc = could not find container \"1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae\": container with ID starting with 1f7dbbfe1d10c29ec94bb0f58f9ff90c360f49653113c67d236bc45ae3215fae not found: ID does not exist"
Oct 30 15:59:05.209233 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.209201    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"]
Oct 30 15:59:05.212016 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.211981    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-9e2b29/premium-w2-9e2b29-kserve-6764dc8c5c-xz5hg"]
Oct 30 15:59:05.243191 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243156    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"]
Oct 30 15:59:05.243816 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243797    4585 cpu_manager.go:414] "containerMap: removing container" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main"
Oct 30 15:59:05.243816 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243815    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main"
Oct 30 15:59:05.243974 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243838    4585 cpu_manager.go:414] "containerMap: removing container" podUID="41f752fd-9135-461f-a763-a5fdb474161f" containerName="revoke-keys"
Oct 30 15:59:05.243974 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243846    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41f752fd-9135-461f-a763-a5fdb474161f" containerName="revoke-keys"
Oct 30 15:59:05.243974 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243875    4585 cpu_manager.go:414] "containerMap: removing container" podUID="5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2" containerName="revoke-keys"
Oct 30 15:59:05.243974 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243883    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2" containerName="revoke-keys"
Oct 30 15:59:05.243974 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243894    4585 cpu_manager.go:414] "containerMap: removing container" podUID="d5ca140b-084b-4889-9881-f28f7dd09b56" containerName="revoke-keys"
Oct 30 15:59:05.243974 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243904    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5ca140b-084b-4889-9881-f28f7dd09b56" containerName="revoke-keys"
Oct 30 15:59:05.243974 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243925    4585 cpu_manager.go:414] "containerMap: removing container" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main"
Oct 30 15:59:05.243974 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.243932    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main"
Oct 30 15:59:05.244385 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.244030    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2" containerName="revoke-keys"
Oct 30 15:59:05.244385 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.244042    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5ca140b-084b-4889-9881-f28f7dd09b56" containerName="revoke-keys"
Oct 30 15:59:05.244385 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.244054    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" containerName="main"
Oct 30 15:59:05.244385 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.244067    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ae2931d-aa72-4b77-bd29-78d29060ff8f" containerName="main"
Oct 30 15:59:05.244385 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.244077    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="41f752fd-9135-461f-a763-a5fdb474161f" containerName="revoke-keys"
Oct 30 15:59:05.249283 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.249259    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.276862 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.276448    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 15:59:05.276862 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.276513    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 15:59:05.276862 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.276555    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 15:59:05.276862 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.276626    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 15:59:05.281325 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.281296    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"]
Oct 30 15:59:05.369806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.366184    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e786081d-rjxfx\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.369806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.366471    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e786081d-rjxfx\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.369806 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.366617    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd4lm\" (UniqueName: \"kubernetes.io/projected/bc457368-452b-4b4b-aab5-9250e11f056e-kube-api-access-rd4lm\") pod \"maas-api-revoke-sub-e786081d-rjxfx\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.467598 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.467545    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd4lm\" (UniqueName: \"kubernetes.io/projected/bc457368-452b-4b4b-aab5-9250e11f056e-kube-api-access-rd4lm\") pod \"maas-api-revoke-sub-e786081d-rjxfx\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.467794 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.467646    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e786081d-rjxfx\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.468275 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.468253    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e786081d-rjxfx\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.468457 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.468436    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e786081d-rjxfx\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.470671 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.470645    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e786081d-rjxfx\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.490953 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.490929    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd4lm\" (UniqueName: \"kubernetes.io/projected/bc457368-452b-4b4b-aab5-9250e11f056e-kube-api-access-rd4lm\") pod \"maas-api-revoke-sub-e786081d-rjxfx\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.576416 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.576379    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:05.720753 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.720670    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg" podUID="2598a7ca-70ff-4bd2-ae89-fa742412fa98" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 15:59:05.720899 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.720682    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4" podUID="547f6ffe-6f5a-4dac-8521-4e92df79f82e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 15:59:05.877942 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:59:05.877896    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc457368_452b_4b4b_aab5_9250e11f056e.slice/crio-3856d2670164ac764c7863b21a0190484d724022e0cf8205d9a303b3982a84ff WatchSource:0}: Error finding container 3856d2670164ac764c7863b21a0190484d724022e0cf8205d9a303b3982a84ff: Status 404 returned error can't find the container with id 3856d2670164ac764c7863b21a0190484d724022e0cf8205d9a303b3982a84ff
Oct 30 15:59:05.878402 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:05.878363    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"]
Oct 30 15:59:06.151253 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:06.151219    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx" event={"ID":"bc457368-452b-4b4b-aab5-9250e11f056e","Type":"ContainerStarted","Data":"1788b6d30b4e5094828ce9a4f506c060d586572c76e0c5a6fdb70b07c48d4c78"}
Oct 30 15:59:06.151430 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:06.151274    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx" event={"ID":"bc457368-452b-4b4b-aab5-9250e11f056e","Type":"ContainerStarted","Data":"3856d2670164ac764c7863b21a0190484d724022e0cf8205d9a303b3982a84ff"}
Oct 30 15:59:06.194602 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:06.194549    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx" podStartSLOduration=1.194535581 podStartE2EDuration="1.194535581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:59:06.191664249 +0000 UTC m=+1255.627268597" watchObservedRunningTime="2026-10-30 15:59:06.194535581 +0000 UTC m=+1255.630139974"
Oct 30 15:59:06.702526 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:06.702492    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44baf777-0b73-472d-8c5f-cc7b693f0409" path="/var/lib/kubelet/pods/44baf777-0b73-472d-8c5f-cc7b693f0409/volumes"
Oct 30 15:59:07.169902 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:07.169863    4585 generic.go:356] "Generic (PLEG): container finished" podID="bc457368-452b-4b4b-aab5-9250e11f056e" containerID="1788b6d30b4e5094828ce9a4f506c060d586572c76e0c5a6fdb70b07c48d4c78" exitCode=0
Oct 30 15:59:07.170132 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:07.170044    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx" event={"ID":"bc457368-452b-4b4b-aab5-9250e11f056e","Type":"ContainerDied","Data":"1788b6d30b4e5094828ce9a4f506c060d586572c76e0c5a6fdb70b07c48d4c78"}
Oct 30 15:59:08.339698 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:08.339672    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:08.411082 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:08.411046    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc457368-452b-4b4b-aab5-9250e11f056e-kube-api-access-rd4lm\" (UniqueName: \"kubernetes.io/projected/bc457368-452b-4b4b-aab5-9250e11f056e-kube-api-access-rd4lm\") pod \"bc457368-452b-4b4b-aab5-9250e11f056e\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") "
Oct 30 15:59:08.411272 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:08.411154    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-openshift-ca\") pod \"bc457368-452b-4b4b-aab5-9250e11f056e\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") "
Oct 30 15:59:08.411272 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:08.411204    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-xks-ca\") pod \"bc457368-452b-4b4b-aab5-9250e11f056e\" (UID: \"bc457368-452b-4b4b-aab5-9250e11f056e\") "
Oct 30 15:59:08.411575 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:08.411550    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-openshift-ca" pod "bc457368-452b-4b4b-aab5-9250e11f056e" (UID: "bc457368-452b-4b4b-aab5-9250e11f056e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:08.411733 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:08.411716    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:08.413960 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:08.413926    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-xks-ca" pod "bc457368-452b-4b4b-aab5-9250e11f056e" (UID: "bc457368-452b-4b4b-aab5-9250e11f056e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:59:08.414061 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:08.413998    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc457368-452b-4b4b-aab5-9250e11f056e-kube-api-access-rd4lm" pod "bc457368-452b-4b4b-aab5-9250e11f056e" (UID: "bc457368-452b-4b4b-aab5-9250e11f056e"). InnerVolumeSpecName "kube-api-access-rd4lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:08.513148 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:08.513058    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd4lm\" (UniqueName: \"kubernetes.io/projected/bc457368-452b-4b4b-aab5-9250e11f056e-kube-api-access-rd4lm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:08.513148 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:08.513091    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bc457368-452b-4b4b-aab5-9250e11f056e-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:09.182929 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:09.182893    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx" event={"ID":"bc457368-452b-4b4b-aab5-9250e11f056e","Type":"ContainerDied","Data":"3856d2670164ac764c7863b21a0190484d724022e0cf8205d9a303b3982a84ff"}
Oct 30 15:59:09.183111 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:09.182932    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"
Oct 30 15:59:09.183111 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:09.182951    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3856d2670164ac764c7863b21a0190484d724022e0cf8205d9a303b3982a84ff"
Oct 30 15:59:12.543137 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.543097    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"]
Oct 30 15:59:12.543690 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.543611    4585 cpu_manager.go:414] "containerMap: removing container" podUID="bc457368-452b-4b4b-aab5-9250e11f056e" containerName="revoke-keys"
Oct 30 15:59:12.543690 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.543625    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc457368-452b-4b4b-aab5-9250e11f056e" containerName="revoke-keys"
Oct 30 15:59:12.543819 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.543749    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc457368-452b-4b4b-aab5-9250e11f056e" containerName="revoke-keys"
Oct 30 15:59:12.548657 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.548632    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.577298 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.577256    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"]
Oct 30 15:59:12.589469 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.589436    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"]
Oct 30 15:59:12.593747 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.593723    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.621050 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.621015    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"]
Oct 30 15:59:12.660512 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.660469    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.660512 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.660525    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.660780 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.660632    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.660780 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.660672    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/809eb8ab-fb39-471e-a770-3eaea55689a9-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.660780 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.660749    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.660919 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.660827    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.660919 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.660877    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.660919 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.660911    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.661025 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.660980    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.661079 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.661038    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.661141 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.661125    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6d9x\" (UniqueName: \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-kube-api-access-l6d9x\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.661173 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.661162    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.661206 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.661189    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.661242 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.661231    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.661313 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.661298    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.661387 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.661343    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.661432 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.661407    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m48vq\" (UniqueName: \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-kube-api-access-m48vq\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.661500 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.661484    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.763027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.762977    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6d9x\" (UniqueName: \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-kube-api-access-l6d9x\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763030    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763306 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763051    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763306 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763091    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763306 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763160    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.763306 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763200    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.763306 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763228    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m48vq\" (UniqueName: \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-kube-api-access-m48vq\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.763306 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763297    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763337    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763377    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763409    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763453    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/809eb8ab-fb39-471e-a770-3eaea55689a9-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763477    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763518    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763533    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763573    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763590    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763619    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.763651 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763655    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.764203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763690    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-credential-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.764203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763713    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.764203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.763822    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.764203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.764040    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.764203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.764100    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-socket\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.764203 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.764179    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-certs\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.764511 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.764291    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-data\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.764764 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.764737    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.764876 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.764765    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/809eb8ab-fb39-471e-a770-3eaea55689a9-istiod-ca-cert\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.766640 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.766613    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.767082 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.767055    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-podinfo\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.767189 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.767168    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.767645 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.767625    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-envoy\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.801792 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.801717    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.801937 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.801898    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6d9x\" (UniqueName: \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-kube-api-access-l6d9x\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:12.802816 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.802795    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m48vq\" (UniqueName: \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-kube-api-access-m48vq\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.810803 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.810779    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-token\") pod \"e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") " pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.865062 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.865027    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:12.907561 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:12.907529    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:13.182581 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:59:13.182551    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50305fbb_13d9_4263_86f9_6ed6b5adf9b3.slice/crio-7750dba4e7c8eda981da36419a3e55e349dfd179150b6f6b9cf91241ed3ed126 WatchSource:0}: Error finding container 7750dba4e7c8eda981da36419a3e55e349dfd179150b6f6b9cf91241ed3ed126: Status 404 returned error can't find the container with id 7750dba4e7c8eda981da36419a3e55e349dfd179150b6f6b9cf91241ed3ed126
Oct 30 15:59:13.182707 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:13.182594    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"]
Oct 30 15:59:13.184964 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:13.184933    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:59:13.185031 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:13.184997    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:59:13.185073 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:13.185027    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:59:13.209228 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:13.209198    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" event={"ID":"50305fbb-13d9-4263-86f9-6ed6b5adf9b3","Type":"ContainerStarted","Data":"7750dba4e7c8eda981da36419a3e55e349dfd179150b6f6b9cf91241ed3ed126"}
Oct 30 15:59:13.263940 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:13.263906    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"]
Oct 30 15:59:13.275486 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:59:13.273599    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809eb8ab_fb39_471e_a770_3eaea55689a9.slice/crio-a059045dcf701083e11ce48558eb60086301b654e9a9d2bf0d16d8684e1c10ad WatchSource:0}: Error finding container a059045dcf701083e11ce48558eb60086301b654e9a9d2bf0d16d8684e1c10ad: Status 404 returned error can't find the container with id a059045dcf701083e11ce48558eb60086301b654e9a9d2bf0d16d8684e1c10ad
Oct 30 15:59:13.277342 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:13.276733    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:59:13.277342 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:13.276801    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:59:13.277342 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:13.276831    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 15:59:14.217944 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.217889    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" event={"ID":"50305fbb-13d9-4263-86f9-6ed6b5adf9b3","Type":"ContainerStarted","Data":"adcdafd246cf0ec79c776b1097f171f65a8a9efb60beb5eb68fc8b725cbf7a96"}
Oct 30 15:59:14.219566 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.219533    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" event={"ID":"809eb8ab-fb39-471e-a770-3eaea55689a9","Type":"ContainerStarted","Data":"d9548d16c12bcfc4480db3c701fc64569c8926c4803e2384a4f7409ac110cbc5"}
Oct 30 15:59:14.219714 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.219585    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" event={"ID":"809eb8ab-fb39-471e-a770-3eaea55689a9","Type":"ContainerStarted","Data":"a059045dcf701083e11ce48558eb60086301b654e9a9d2bf0d16d8684e1c10ad"}
Oct 30 15:59:14.303790 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.303724    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podStartSLOduration=2.3037082509999998 podStartE2EDuration="2.303708251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:59:14.303555421 +0000 UTC m=+1263.739159770" watchObservedRunningTime="2026-10-30 15:59:14.303708251 +0000 UTC m=+1263.739312599"
Oct 30 15:59:14.377008 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.362094    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podStartSLOduration=2.362074423 podStartE2EDuration="2.362074423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:59:14.360925949 +0000 UTC m=+1263.796530299" watchObservedRunningTime="2026-10-30 15:59:14.362074423 +0000 UTC m=+1263.797678770"
Oct 30 15:59:14.868114 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.868070    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:14.868528 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.868500    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 15:59:14.868738 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.868711    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 15:59:14.917064 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.917029    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 15:59:14.917386 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.917329    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 15:59:14.917890 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:14.917869    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:15.738373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:15.736467    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4" podUID="547f6ffe-6f5a-4dac-8521-4e92df79f82e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 15:59:15.745551 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:15.745498    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg" podUID="2598a7ca-70ff-4bd2-ae89-fa742412fa98" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 15:59:15.910374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:15.884082    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 15:59:15.910374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:15.884143    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 15:59:15.961549 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:15.961514    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 15:59:15.962263 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:15.961818    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 15:59:16.026442 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:16.013632    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:59:16.869624 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:16.856804    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"]
Oct 30 15:59:16.870148 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:16.869877    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 15:59:16.870148 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:16.869978    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 15:59:16.883752 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:16.878821    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a2b3f1-nd7wm"]
Oct 30 15:59:16.883752 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:16.883536    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:16.915967 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:16.911438    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 15:59:16.915967 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:16.911500    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 15:59:17.030407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:17.028799    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:17.030407 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:17.029745    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:59:17.885093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:17.866841    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 15:59:17.885093 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:17.866922    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 15:59:17.885708 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:17.885124    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:17.885708 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:17.885646    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:59:17.920908 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:17.920865    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 15:59:17.921198 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:17.921160    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 15:59:17.994652 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:17.993452    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:18.051373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:18.036898    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:59:18.212825 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:59:18.212779    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:59:18.892962 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:18.892913    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": context deadline exceeded"
Oct 30 15:59:18.972568 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:18.972526    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 15:59:18.986719 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:18.986660    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 15:59:18.987944 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:18.987899    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 15:59:18.996060 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:18.996014    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 15:59:19.033898 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:19.033843    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:19.036004 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:19.035488    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90c831a0-898d-479e-8c5e-c7170b97bef7" path="/var/lib/kubelet/pods/90c831a0-898d-479e-8c5e-c7170b97bef7/volumes"
Oct 30 15:59:19.042236 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:19.042204    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:59:19.068632 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:19.068227    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:59:19.889553 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:19.889518    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 15:59:19.914372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:19.889823    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 15:59:19.920484 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:19.916198    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 15:59:19.931391 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:19.924453    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 15:59:19.956638 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:19.956525    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq" podUID="84e32e1d-bae5-4233-ac38-50ff6ccaa774" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:20.058372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.056243    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:59:20.193370 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.090637    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 15:59:20.193980 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.120495    4585 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:20.194135 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.186878    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:59:20.194251 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.194225    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:20.560791 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.560706    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:59:20.561085 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.561018    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:20.681372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.671795    4585 patch_prober.go:29] interesting pod/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:59:20.681372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.671861    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:20.681372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.673599    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq"
Oct 30 15:59:20.683605 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.683563    4585 generic.go:356] "Generic (PLEG): container finished" podID="2598a7ca-70ff-4bd2-ae89-fa742412fa98" containerID="239bb281af2a6ddefa6abe67a56aa4ed1c5d508068bee9d2a3886ff13ad0029e" exitCode=0
Oct 30 15:59:20.683945 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.683791    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg" event={"ID":"2598a7ca-70ff-4bd2-ae89-fa742412fa98","Type":"ContainerDied","Data":"239bb281af2a6ddefa6abe67a56aa4ed1c5d508068bee9d2a3886ff13ad0029e"}
Oct 30 15:59:20.866831 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.866736    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 15:59:20.867112 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:20.867077    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 15:59:21.007372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.002511    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 15:59:21.007372 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.002573    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 15:59:21.079612 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.079562    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 15:59:21.137042 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.136952    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:59:21.513606 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.513574    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:59:21.517768 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.517741    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:59:21.705787 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.705702    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2598a7ca-70ff-4bd2-ae89-fa742412fa98-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2598a7ca-70ff-4bd2-ae89-fa742412fa98-plugins-config-volume\") pod \"2598a7ca-70ff-4bd2-ae89-fa742412fa98\" (UID: \"2598a7ca-70ff-4bd2-ae89-fa742412fa98\") "
Oct 30 15:59:21.706068 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.706049    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2598a7ca-70ff-4bd2-ae89-fa742412fa98-kube-api-access-xvhv9\" (UniqueName: \"kubernetes.io/projected/2598a7ca-70ff-4bd2-ae89-fa742412fa98-kube-api-access-xvhv9\") pod \"2598a7ca-70ff-4bd2-ae89-fa742412fa98\" (UID: \"2598a7ca-70ff-4bd2-ae89-fa742412fa98\") "
Oct 30 15:59:21.707502 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.707478    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2598a7ca-70ff-4bd2-ae89-fa742412fa98-plugins-config-volume" pod "2598a7ca-70ff-4bd2-ae89-fa742412fa98" (UID: "2598a7ca-70ff-4bd2-ae89-fa742412fa98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:21.715421 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.715394    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2598a7ca-70ff-4bd2-ae89-fa742412fa98-kube-api-access-xvhv9" pod "2598a7ca-70ff-4bd2-ae89-fa742412fa98" (UID: "2598a7ca-70ff-4bd2-ae89-fa742412fa98"). InnerVolumeSpecName "kube-api-access-xvhv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:21.781684 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.781654    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:59:21.814373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.810140    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvhv9\" (UniqueName: \"kubernetes.io/projected/2598a7ca-70ff-4bd2-ae89-fa742412fa98-kube-api-access-xvhv9\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:21.814629 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.814605    4585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2598a7ca-70ff-4bd2-ae89-fa742412fa98-plugins-config-volume\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:21.815863 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.815832    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg" event={"ID":"2598a7ca-70ff-4bd2-ae89-fa742412fa98","Type":"ContainerDied","Data":"bafc351362365e3432ca256a886298a23bbcb87eaac943f24a6696691d0d634d"}
Oct 30 15:59:21.816079 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.816064    4585 scope.go:122] "RemoveContainer" containerID="239bb281af2a6ddefa6abe67a56aa4ed1c5d508068bee9d2a3886ff13ad0029e"
Oct 30 15:59:21.816366 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.816334    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"
Oct 30 15:59:21.827249 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.825235    4585 generic.go:356] "Generic (PLEG): container finished" podID="547f6ffe-6f5a-4dac-8521-4e92df79f82e" containerID="92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f" exitCode=0
Oct 30 15:59:21.827510 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.825388    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4" event={"ID":"547f6ffe-6f5a-4dac-8521-4e92df79f82e","Type":"ContainerDied","Data":"92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f"}
Oct 30 15:59:21.832403 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.825421    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"
Oct 30 15:59:21.862374 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.832900    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4" event={"ID":"547f6ffe-6f5a-4dac-8521-4e92df79f82e","Type":"ContainerDied","Data":"48fa8743fea9252503460527b21b4082f4c8614f14c87e50a0351e7df3fc9765"}
Oct 30 15:59:21.926545 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.926512    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/547f6ffe-6f5a-4dac-8521-4e92df79f82e-kube-api-access-74988\" (UniqueName: \"kubernetes.io/projected/547f6ffe-6f5a-4dac-8521-4e92df79f82e-kube-api-access-74988\") pod \"547f6ffe-6f5a-4dac-8521-4e92df79f82e\" (UID: \"547f6ffe-6f5a-4dac-8521-4e92df79f82e\") "
Oct 30 15:59:21.926900 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.926886    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/547f6ffe-6f5a-4dac-8521-4e92df79f82e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547f6ffe-6f5a-4dac-8521-4e92df79f82e-plugins-config-volume\") pod \"547f6ffe-6f5a-4dac-8521-4e92df79f82e\" (UID: \"547f6ffe-6f5a-4dac-8521-4e92df79f82e\") "
Oct 30 15:59:21.927660 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.927629    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/547f6ffe-6f5a-4dac-8521-4e92df79f82e-plugins-config-volume" pod "547f6ffe-6f5a-4dac-8521-4e92df79f82e" (UID: "547f6ffe-6f5a-4dac-8521-4e92df79f82e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:21.935925 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.935883    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 15:59:21.963263 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.962601    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547f6ffe-6f5a-4dac-8521-4e92df79f82e-kube-api-access-74988" pod "547f6ffe-6f5a-4dac-8521-4e92df79f82e" (UID: "547f6ffe-6f5a-4dac-8521-4e92df79f82e"). InnerVolumeSpecName "kube-api-access-74988". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:21.963619 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.963597    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 15:59:21.963842 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.963794    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 15:59:21.964078 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:21.963957    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 15:59:22.001816 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.001786    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"]
Oct 30 15:59:22.004931 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.004907    4585 scope.go:122] "RemoveContainer" containerID="92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f"
Oct 30 15:59:22.005814 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.005789    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-a2b3f1-5c87b89f9f-pvbfg"]
Oct 30 15:59:22.014891 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.014876    4585 scope.go:122] "RemoveContainer" containerID="92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f"
Oct 30 15:59:22.015192 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:59:22.015113    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f\": container with ID starting with 92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f not found: ID does not exist" containerID="92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f"
Oct 30 15:59:22.015192 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.015158    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f"} err="failed to get container status \"92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f\": rpc error: code = NotFound desc = could not find container \"92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f\": container with ID starting with 92f50f8a7f15a702abdf676d7822f1b359e9b8eae33014edf7896f5614de5b1f not found: ID does not exist"
Oct 30 15:59:22.037373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.028570    4585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547f6ffe-6f5a-4dac-8521-4e92df79f82e-plugins-config-volume\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:22.037373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.028596    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74988\" (UniqueName: \"kubernetes.io/projected/547f6ffe-6f5a-4dac-8521-4e92df79f82e-kube-api-access-74988\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:22.225418 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.225306    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"]
Oct 30 15:59:22.256380 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.255837    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9e2b29-6cd4b99f67-d2vx4"]
Oct 30 15:59:22.703779 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.703745    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2598a7ca-70ff-4bd2-ae89-fa742412fa98" path="/var/lib/kubelet/pods/2598a7ca-70ff-4bd2-ae89-fa742412fa98/volumes"
Oct 30 15:59:22.704165 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.704151    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="547f6ffe-6f5a-4dac-8521-4e92df79f82e" path="/var/lib/kubelet/pods/547f6ffe-6f5a-4dac-8521-4e92df79f82e/volumes"
Oct 30 15:59:22.871389 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.870488    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 15:59:22.871389 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.870552    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 15:59:22.908464 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.908424    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 15:59:22.908644 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:22.908477    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 15:59:23.866073 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:23.866030    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 15:59:23.866521 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:23.866101    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 15:59:23.908586 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:23.908545    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 15:59:23.908763 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:23.908616    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 15:59:25.911568 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:25.911487    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:59:25.911568 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:25.911557    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:25.951027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:25.950450    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:59:25.951027 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:25.950515    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:26.747450 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.747072    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"]
Oct 30 15:59:26.747743 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.747724    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"]
Oct 30 15:59:26.747867 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.747855    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"]
Oct 30 15:59:26.747984 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.747974    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"]
Oct 30 15:59:26.748274 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.748247    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy" containerID="cri-o://b376c7495326fa6c5273b37f2233d667c4436cebb002db40c05f17e7c26042bf" gracePeriod=30
Oct 30 15:59:26.748871 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.748841    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerName="istio-proxy" containerID="cri-o://a8c23bb0aefb864400ec39cb9acdf63501934ca9e7472462eee2d2721861eafa" gracePeriod=30
Oct 30 15:59:26.749196 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.749174    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy" containerID="cri-o://a3a54a84938ec4ee44391cb13e95ce96143d4059f245fb8858436c133faaf1fa" gracePeriod=30
Oct 30 15:59:26.749510 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.749487    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy" containerID="cri-o://65269abff28a2f3289a78852ed93e851e135149e0cd298d24093560c9c72b07f" gracePeriod=30
Oct 30 15:59:26.751137 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.751120    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:26.752275 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.752260    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:26.754137 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.754122    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 15:59:26.960511 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.960468    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 15:59:26.961227 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.961197    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 15:59:26.963204 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.963180    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:26.964521 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:26.964503    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:27.007141 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:27.007037    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 15:59:27.482781 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:27.482732    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"]
Oct 30 15:59:27.483649 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:27.483598    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy" containerID="cri-o://70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce" gracePeriod=30
Oct 30 15:59:31.717721 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:31.717684    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"]
Oct 30 15:59:31.718199 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:31.717925    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy" containerID="cri-o://7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502" gracePeriod=30
Oct 30 15:59:31.756322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:31.756273    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"]
Oct 30 15:59:31.756586 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:31.756561    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy" containerID="cri-o://928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd" gracePeriod=30
Oct 30 15:59:31.760873 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:31.760844    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"]
Oct 30 15:59:31.761611 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:31.761581    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerName="istio-proxy" containerID="cri-o://888d9afa53ca835e0e432cb1662b5087e01fd6be46c3c4e07d9b58b6701ad720" gracePeriod=30
Oct 30 15:59:32.005902 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.005672    4585 generic.go:356] "Generic (PLEG): container finished" podID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerID="a8c23bb0aefb864400ec39cb9acdf63501934ca9e7472462eee2d2721861eafa" exitCode=0
Oct 30 15:59:32.005902 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.005781    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" event={"ID":"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5","Type":"ContainerDied","Data":"a8c23bb0aefb864400ec39cb9acdf63501934ca9e7472462eee2d2721861eafa"}
Oct 30 15:59:32.008202 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.008045    4585 generic.go:356] "Generic (PLEG): container finished" podID="9400972d-230d-4521-a683-bfc605f3b647" containerID="b376c7495326fa6c5273b37f2233d667c4436cebb002db40c05f17e7c26042bf" exitCode=0
Oct 30 15:59:32.008202 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.008138    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" event={"ID":"9400972d-230d-4521-a683-bfc605f3b647","Type":"ContainerDied","Data":"b376c7495326fa6c5273b37f2233d667c4436cebb002db40c05f17e7c26042bf"}
Oct 30 15:59:32.010608 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.010447    4585 generic.go:356] "Generic (PLEG): container finished" podID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerID="a3a54a84938ec4ee44391cb13e95ce96143d4059f245fb8858436c133faaf1fa" exitCode=0
Oct 30 15:59:32.010608 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.010546    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" event={"ID":"cd1d3da4-a476-4f37-b2c6-acc545bfd734","Type":"ContainerDied","Data":"a3a54a84938ec4ee44391cb13e95ce96143d4059f245fb8858436c133faaf1fa"}
Oct 30 15:59:32.012802 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.012638    4585 generic.go:356] "Generic (PLEG): container finished" podID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerID="65269abff28a2f3289a78852ed93e851e135149e0cd298d24093560c9c72b07f" exitCode=0
Oct 30 15:59:32.012802 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.012715    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" event={"ID":"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb","Type":"ContainerDied","Data":"65269abff28a2f3289a78852ed93e851e135149e0cd298d24093560c9c72b07f"}
Oct 30 15:59:32.099622 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.099391    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:59:32.143047 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.142997    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-podinfo\") pod \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") "
Oct 30 15:59:32.143246 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.143143    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istiod-ca-cert\") pod \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") "
Oct 30 15:59:32.143246 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.143241    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-token\" (UniqueName: \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-token\") pod \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") "
Oct 30 15:59:32.143390 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.143285    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-certs\") pod \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") "
Oct 30 15:59:32.143390 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.143338    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-data\") pod \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") "
Oct 30 15:59:32.143488 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.143388    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-kube-api-access-7jc5j\" (UniqueName: \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-kube-api-access-7jc5j\") pod \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") "
Oct 30 15:59:32.143488 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.143438    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-socket\") pod \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") "
Oct 30 15:59:32.143488 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.143470    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-envoy\") pod \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") "
Oct 30 15:59:32.143706 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.143528    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-credential-socket\") pod \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\" (UID: \"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5\") "
Oct 30 15:59:32.144843 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.144239    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-credential-socket" pod "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" (UID: "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.144843 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.144309    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istiod-ca-cert" pod "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" (UID: "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:32.144843 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.144495    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-socket" pod "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" (UID: "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.144843 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.144664    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-data" pod "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" (UID: "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.144843 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.144774    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-certs" pod "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" (UID: "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.147915 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.147881    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-kube-api-access-7jc5j" pod "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" (UID: "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5"). InnerVolumeSpecName "kube-api-access-7jc5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:32.148267 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.148242    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-envoy" pod "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" (UID: "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.149945 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.149917    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-token" pod "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" (UID: "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:32.150936 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.150914    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-podinfo" pod "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" (UID: "4db0f08f-678a-4f7e-99ee-8c7bb4de5af5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:59:32.162449 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.162423    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.245139    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-credential-socket\") pod \"9400972d-230d-4521-a683-bfc605f3b647\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") "
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.245270    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-certs\") pod \"9400972d-230d-4521-a683-bfc605f3b647\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") "
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.245322    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-istio-token\" (UniqueName: \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-istio-token\") pod \"9400972d-230d-4521-a683-bfc605f3b647\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") "
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.245387    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-data\") pod \"9400972d-230d-4521-a683-bfc605f3b647\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") "
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.245445    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-socket\") pod \"9400972d-230d-4521-a683-bfc605f3b647\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") "
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.245566    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-envoy\") pod \"9400972d-230d-4521-a683-bfc605f3b647\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") "
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.245594    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9400972d-230d-4521-a683-bfc605f3b647-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9400972d-230d-4521-a683-bfc605f3b647-istiod-ca-cert\") pod \"9400972d-230d-4521-a683-bfc605f3b647\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") "
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.245853    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9400972d-230d-4521-a683-bfc605f3b647-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9400972d-230d-4521-a683-bfc605f3b647-istio-podinfo\") pod \"9400972d-230d-4521-a683-bfc605f3b647\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") "
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.245950    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-kube-api-access-wlbjx\" (UniqueName: \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-kube-api-access-wlbjx\") pod \"9400972d-230d-4521-a683-bfc605f3b647\" (UID: \"9400972d-230d-4521-a683-bfc605f3b647\") "
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.246561    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.246580    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.246613    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.246628    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.246642    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.246654    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.246676    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jc5j\" (UniqueName: \"kubernetes.io/projected/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-kube-api-access-7jc5j\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.246692    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.246706    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.248168    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-socket" pod "9400972d-230d-4521-a683-bfc605f3b647" (UID: "9400972d-230d-4521-a683-bfc605f3b647"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.248394    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-credential-socket" pod "9400972d-230d-4521-a683-bfc605f3b647" (UID: "9400972d-230d-4521-a683-bfc605f3b647"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.248540    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9400972d-230d-4521-a683-bfc605f3b647-istiod-ca-cert" pod "9400972d-230d-4521-a683-bfc605f3b647" (UID: "9400972d-230d-4521-a683-bfc605f3b647"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.248588    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-certs" pod "9400972d-230d-4521-a683-bfc605f3b647" (UID: "9400972d-230d-4521-a683-bfc605f3b647"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.248641    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-data" pod "9400972d-230d-4521-a683-bfc605f3b647" (UID: "9400972d-230d-4521-a683-bfc605f3b647"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.253373 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.250382    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9400972d-230d-4521-a683-bfc605f3b647-istio-podinfo" pod "9400972d-230d-4521-a683-bfc605f3b647" (UID: "9400972d-230d-4521-a683-bfc605f3b647"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:59:32.256762 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.256680    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-istio-token" pod "9400972d-230d-4521-a683-bfc605f3b647" (UID: "9400972d-230d-4521-a683-bfc605f3b647"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:32.256762 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.256697    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-kube-api-access-wlbjx" pod "9400972d-230d-4521-a683-bfc605f3b647" (UID: "9400972d-230d-4521-a683-bfc605f3b647"). InnerVolumeSpecName "kube-api-access-wlbjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:32.257952 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.257916    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-envoy" pod "9400972d-230d-4521-a683-bfc605f3b647" (UID: "9400972d-230d-4521-a683-bfc605f3b647"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.349337 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.349289    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.349337 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.349341    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.349570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.349375    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9400972d-230d-4521-a683-bfc605f3b647-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.349570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.349388    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9400972d-230d-4521-a683-bfc605f3b647-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.349570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.349432    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlbjx\" (UniqueName: \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-kube-api-access-wlbjx\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.349570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.349448    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.349570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.349462    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.349570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.349476    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9400972d-230d-4521-a683-bfc605f3b647-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.349570 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.349488    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9400972d-230d-4521-a683-bfc605f3b647-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.496635 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.496261    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:59:32.554953 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.554908    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-kube-api-access-l2zp6\" (UniqueName: \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-kube-api-access-l2zp6\") pod \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") "
Oct 30 15:59:32.554953 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.554956    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-token\" (UniqueName: \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-token\") pod \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") "
Oct 30 15:59:32.555237 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.554983    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-data\") pod \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") "
Oct 30 15:59:32.555237 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.555056    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-podinfo\") pod \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") "
Oct 30 15:59:32.555237 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.555157    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istiod-ca-cert\") pod \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") "
Oct 30 15:59:32.555505 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.555240    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-certs\") pod \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") "
Oct 30 15:59:32.555505 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.555276    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-socket\") pod \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") "
Oct 30 15:59:32.555505 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.555312    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-credential-socket\") pod \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") "
Oct 30 15:59:32.555505 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.555412    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-envoy\") pod \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\" (UID: \"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb\") "
Oct 30 15:59:32.556147 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.556055    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-certs" pod "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" (UID: "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.556756 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.556729    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-socket" pod "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" (UID: "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.557021 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.556979    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-credential-socket" pod "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" (UID: "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.557214 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.557174    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-data" pod "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" (UID: "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.557587 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.557557    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istiod-ca-cert" pod "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" (UID: "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:32.558636 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.558580    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-podinfo" pod "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" (UID: "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:59:32.559181 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.559145    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-token" pod "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" (UID: "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:32.559688 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.559662    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-kube-api-access-l2zp6" pod "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" (UID: "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb"). InnerVolumeSpecName "kube-api-access-l2zp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:32.559781 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.559755    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-envoy" pod "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" (UID: "dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.603273 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.603239    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.656984    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-credential-socket\") pod \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") "
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.657056    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istiod-ca-cert\") pod \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") "
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.657129    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-token\" (UniqueName: \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-token\") pod \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") "
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.657181    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-podinfo\") pod \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") "
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.657282    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-envoy\") pod \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") "
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.657324    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-data\") pod \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") "
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.657390    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-socket\") pod \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") "
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.657474    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-certs\") pod \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") "
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.657506    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-kube-api-access-2bdj2\" (UniqueName: \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-kube-api-access-2bdj2\") pod \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\" (UID: \"cd1d3da4-a476-4f37-b2c6-acc545bfd734\") "
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.658024    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.658045    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.658061    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.658076    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.658090    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.658103    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.658118    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2zp6\" (UniqueName: \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-kube-api-access-l2zp6\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.658132    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.658539 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.658146    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.659643 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.659041    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-credential-socket" pod "cd1d3da4-a476-4f37-b2c6-acc545bfd734" (UID: "cd1d3da4-a476-4f37-b2c6-acc545bfd734"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.660529 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.659862    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-data" pod "cd1d3da4-a476-4f37-b2c6-acc545bfd734" (UID: "cd1d3da4-a476-4f37-b2c6-acc545bfd734"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.660529 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.660318    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-socket" pod "cd1d3da4-a476-4f37-b2c6-acc545bfd734" (UID: "cd1d3da4-a476-4f37-b2c6-acc545bfd734"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.660529 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.660468    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-certs" pod "cd1d3da4-a476-4f37-b2c6-acc545bfd734" (UID: "cd1d3da4-a476-4f37-b2c6-acc545bfd734"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.660769 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.660547    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istiod-ca-cert" pod "cd1d3da4-a476-4f37-b2c6-acc545bfd734" (UID: "cd1d3da4-a476-4f37-b2c6-acc545bfd734"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:32.660868 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.660830    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-kube-api-access-2bdj2" pod "cd1d3da4-a476-4f37-b2c6-acc545bfd734" (UID: "cd1d3da4-a476-4f37-b2c6-acc545bfd734"). InnerVolumeSpecName "kube-api-access-2bdj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:32.663301 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.663276    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-envoy" pod "cd1d3da4-a476-4f37-b2c6-acc545bfd734" (UID: "cd1d3da4-a476-4f37-b2c6-acc545bfd734"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.664410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.664317    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-token" pod "cd1d3da4-a476-4f37-b2c6-acc545bfd734" (UID: "cd1d3da4-a476-4f37-b2c6-acc545bfd734"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:32.670568 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.668119    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-podinfo" pod "cd1d3da4-a476-4f37-b2c6-acc545bfd734" (UID: "cd1d3da4-a476-4f37-b2c6-acc545bfd734"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:59:32.759971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.759855    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.759971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.759891    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.759971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.759906    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.759971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.759917    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.759971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.759929    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.759971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.759943    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.759971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.759954    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.759971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.759963    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd1d3da4-a476-4f37-b2c6-acc545bfd734-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.759971 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.759973    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bdj2\" (UniqueName: \"kubernetes.io/projected/cd1d3da4-a476-4f37-b2c6-acc545bfd734-kube-api-access-2bdj2\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.805000 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.804975    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:59:32.861183 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861149    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-certs\") pod \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") "
Oct 30 15:59:32.861412 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861202    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-podinfo\") pod \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") "
Oct 30 15:59:32.861412 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861227    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-data\") pod \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") "
Oct 30 15:59:32.861412 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861248    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-socket\") pod \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") "
Oct 30 15:59:32.861412 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861269    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-envoy\") pod \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") "
Oct 30 15:59:32.861642 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861505    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-kube-api-access-q6ncj\" (UniqueName: \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-kube-api-access-q6ncj\") pod \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") "
Oct 30 15:59:32.861642 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861608    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-credential-socket\") pod \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") "
Oct 30 15:59:32.861742 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861550    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-certs" pod "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" (UID: "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.861742 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861563    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-data" pod "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" (UID: "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.861742 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861671    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-token\" (UniqueName: \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-token\") pod \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") "
Oct 30 15:59:32.861742 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861726    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-socket" pod "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" (UID: "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.861949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861771    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istiod-ca-cert\") pod \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\" (UID: \"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2\") "
Oct 30 15:59:32.861949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.861836    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-credential-socket" pod "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" (UID: "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.862236 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.862186    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istiod-ca-cert" pod "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" (UID: "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:32.862410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.862313    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.862410 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.862335    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.862599 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.862364    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.862599 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.862596    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.864477 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.862612    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.868754 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.868703    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-kube-api-access-q6ncj" pod "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" (UID: "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2"). InnerVolumeSpecName "kube-api-access-q6ncj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:32.869516 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.869479    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-envoy" pod "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" (UID: "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:32.870056 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.869597    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-podinfo" pod "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" (UID: "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:59:32.870056 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.869694    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-token" pod "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" (UID: "5a0bee82-0d1d-4bb9-8575-3683fe99d5f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:32.965972 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.965935    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.965972 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.965974    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.966193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.965990    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:32.966193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:32.966006    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6ncj\" (UniqueName: \"kubernetes.io/projected/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2-kube-api-access-q6ncj\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:33.020899 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.020776    4585 generic.go:356] "Generic (PLEG): container finished" podID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerID="70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce" exitCode=0
Oct 30 15:59:33.020899 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.020842    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"
Oct 30 15:59:33.021124 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.020960    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" event={"ID":"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2","Type":"ContainerDied","Data":"70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce"}
Oct 30 15:59:33.021124 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.021030    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v" event={"ID":"5a0bee82-0d1d-4bb9-8575-3683fe99d5f2","Type":"ContainerDied","Data":"44752e70f0f002574159318e10a3bd2164b5a55669850df47b7bda06d22bb811"}
Oct 30 15:59:33.021124 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.021059    4585 scope.go:122] "RemoveContainer" containerID="70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce"
Oct 30 15:59:33.023949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.023915    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"
Oct 30 15:59:33.023949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.023926    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl" event={"ID":"4db0f08f-678a-4f7e-99ee-8c7bb4de5af5","Type":"ContainerDied","Data":"d7ae63ca5780745e73b7ba8c01496c44bb14064b4d58ba0aed1c19bf5e2cb873"}
Oct 30 15:59:33.026564 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.026544    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"
Oct 30 15:59:33.027305 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.026551    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj" event={"ID":"9400972d-230d-4521-a683-bfc605f3b647","Type":"ContainerDied","Data":"4eed71234431aba7409af352151ac685d0aba5f25c48d8b83afba1e099c0d57d"}
Oct 30 15:59:33.029155 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.028586    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z" event={"ID":"cd1d3da4-a476-4f37-b2c6-acc545bfd734","Type":"ContainerDied","Data":"d01907a54e532cd67f4ca70c50f1a568895e6fbadcb2f45b2e75dd57f5b7a928"}
Oct 30 15:59:33.029155 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.028682    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"
Oct 30 15:59:33.036145 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.036124    4585 scope.go:122] "RemoveContainer" containerID="70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce"
Oct 30 15:59:33.036636 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:59:33.036607    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce\": container with ID starting with 70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce not found: ID does not exist" containerID="70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce"
Oct 30 15:59:33.036780 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.036755    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce"} err="failed to get container status \"70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce\": rpc error: code = NotFound desc = could not find container \"70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce\": container with ID starting with 70a79e7767be58225cecdacf10751622e5a4302d3a8e63c4850d7c435d9a29ce not found: ID does not exist"
Oct 30 15:59:33.036876 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.036862    4585 scope.go:122] "RemoveContainer" containerID="a8c23bb0aefb864400ec39cb9acdf63501934ca9e7472462eee2d2721861eafa"
Oct 30 15:59:33.036955 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.036780    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs" event={"ID":"dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb","Type":"ContainerDied","Data":"db0af2f30a38ff24eecaf87f02c9ae164bcb44a783881b961e956b0f6b46f068"}
Oct 30 15:59:33.036999 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.036788    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"
Oct 30 15:59:33.049982 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.049884    4585 scope.go:122] "RemoveContainer" containerID="b376c7495326fa6c5273b37f2233d667c4436cebb002db40c05f17e7c26042bf"
Oct 30 15:59:33.063031 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.063005    4585 scope.go:122] "RemoveContainer" containerID="a3a54a84938ec4ee44391cb13e95ce96143d4059f245fb8858436c133faaf1fa"
Oct 30 15:59:33.075378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.075322    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"]
Oct 30 15:59:33.076829 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.076806    4585 scope.go:122] "RemoveContainer" containerID="65269abff28a2f3289a78852ed93e851e135149e0cd298d24093560c9c72b07f"
Oct 30 15:59:33.078746 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.078722    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-4xwzl"]
Oct 30 15:59:33.110398 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.110368    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"]
Oct 30 15:59:33.114126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.114102    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-040f9a-openshift-default-6bd6df64dc-tjx9v"]
Oct 30 15:59:33.163437 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.163401    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"]
Oct 30 15:59:33.170546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.170513    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-frr4z"]
Oct 30 15:59:33.208553 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.208505    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"]
Oct 30 15:59:33.213109 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.213082    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-v5wgj"]
Oct 30 15:59:33.259392 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.259357    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"]
Oct 30 15:59:33.259549 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:33.259416    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a2b3f1-openshift-default-57587c5fc-h6xxs"]
Oct 30 15:59:34.702502 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:34.702471    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" path="/var/lib/kubelet/pods/4db0f08f-678a-4f7e-99ee-8c7bb4de5af5/volumes"
Oct 30 15:59:34.702939 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:34.702925    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" path="/var/lib/kubelet/pods/5a0bee82-0d1d-4bb9-8575-3683fe99d5f2/volumes"
Oct 30 15:59:34.703325 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:34.703313    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9400972d-230d-4521-a683-bfc605f3b647" path="/var/lib/kubelet/pods/9400972d-230d-4521-a683-bfc605f3b647/volumes"
Oct 30 15:59:34.704033 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:34.704018    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" path="/var/lib/kubelet/pods/cd1d3da4-a476-4f37-b2c6-acc545bfd734/volumes"
Oct 30 15:59:34.704501 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:34.704486    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" path="/var/lib/kubelet/pods/dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb/volumes"
Oct 30 15:59:35.034586 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:35.034495    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 15:59:35.034586 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:35.034566    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 15:59:36.160969 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:36.160927    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 15:59:36.161384 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:36.160989    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 15:59:36.862362 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:36.862319    4585 patch_prober.go:29] interesting pod/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 15:59:36.862458 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:36.862386    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 15:59:37.045647 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.045620    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:59:37.059297 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.059258    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:59:37.067126 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.067102    4585 generic.go:356] "Generic (PLEG): container finished" podID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerID="888d9afa53ca835e0e432cb1662b5087e01fd6be46c3c4e07d9b58b6701ad720" exitCode=0
Oct 30 15:59:37.067244 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.067174    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" event={"ID":"8e084541-731b-46dc-a2e0-dd989b1b36a6","Type":"ContainerDied","Data":"888d9afa53ca835e0e432cb1662b5087e01fd6be46c3c4e07d9b58b6701ad720"}
Oct 30 15:59:37.068652 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.068628    4585 generic.go:356] "Generic (PLEG): container finished" podID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerID="928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd" exitCode=0
Oct 30 15:59:37.068794 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.068675    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"
Oct 30 15:59:37.068794 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.068714    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" event={"ID":"69f16429-e079-4aa7-adba-a8ceecd65b1f","Type":"ContainerDied","Data":"928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd"}
Oct 30 15:59:37.068794 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.068788    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk" event={"ID":"69f16429-e079-4aa7-adba-a8ceecd65b1f","Type":"ContainerDied","Data":"9cb3336880b99dbb0685f248ab552683701ea8f497a166d0cf014778e6bbb356"}
Oct 30 15:59:37.068912 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.068817    4585 scope.go:122] "RemoveContainer" containerID="928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd"
Oct 30 15:59:37.070677 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.070638    4585 generic.go:356] "Generic (PLEG): container finished" podID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerID="7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502" exitCode=0
Oct 30 15:59:37.070805 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.070789    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" event={"ID":"a026ba91-cf65-4376-b0e2-386a4bd888ef","Type":"ContainerDied","Data":"7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502"}
Oct 30 15:59:37.070891 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.070835    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx" event={"ID":"a026ba91-cf65-4376-b0e2-386a4bd888ef","Type":"ContainerDied","Data":"6c4cc982e0a1860a1bc58dd236c4775622aa05b34f2e3bfba5f6f47fc15e3d42"}
Oct 30 15:59:37.070956 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.070911    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"
Oct 30 15:59:37.080195 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.080174    4585 scope.go:122] "RemoveContainer" containerID="928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd"
Oct 30 15:59:37.080585 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:59:37.080559    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd\": container with ID starting with 928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd not found: ID does not exist" containerID="928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd"
Oct 30 15:59:37.080687 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.080599    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd"} err="failed to get container status \"928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd\": rpc error: code = NotFound desc = could not find container \"928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd\": container with ID starting with 928496fb01f37b503dcee5b52b03baf6ba4f6ff5aa266de03d5fbf88bc546dbd not found: ID does not exist"
Oct 30 15:59:37.080687 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.080625    4585 scope.go:122] "RemoveContainer" containerID="7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502"
Oct 30 15:59:37.090541 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.090520    4585 scope.go:122] "RemoveContainer" containerID="7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502"
Oct 30 15:59:37.090868 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:59:37.090840    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502\": container with ID starting with 7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502 not found: ID does not exist" containerID="7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502"
Oct 30 15:59:37.090949 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.090880    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502"} err="failed to get container status \"7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502\": rpc error: code = NotFound desc = could not find container \"7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502\": container with ID starting with 7b0975580e584cd517734b4fde6b2d47df99e8ebf2df660e1180960ad227b502 not found: ID does not exist"
Oct 30 15:59:37.110192 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.110160    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:59:37.113919 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.113886    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-socket\") pod \"a026ba91-cf65-4376-b0e2-386a4bd888ef\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") "
Oct 30 15:59:37.114029 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.113962    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-envoy\") pod \"a026ba91-cf65-4376-b0e2-386a4bd888ef\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") "
Oct 30 15:59:37.114029 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.114002    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-certs\") pod \"a026ba91-cf65-4376-b0e2-386a4bd888ef\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") "
Oct 30 15:59:37.114124 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.114035    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-token\" (UniqueName: \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-token\") pod \"a026ba91-cf65-4376-b0e2-386a4bd888ef\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") "
Oct 30 15:59:37.114124 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.114100    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-kube-api-access-kd87h\" (UniqueName: \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-kube-api-access-kd87h\") pod \"a026ba91-cf65-4376-b0e2-386a4bd888ef\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") "
Oct 30 15:59:37.114200 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.114164    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a026ba91-cf65-4376-b0e2-386a4bd888ef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a026ba91-cf65-4376-b0e2-386a4bd888ef-istiod-ca-cert\") pod \"a026ba91-cf65-4376-b0e2-386a4bd888ef\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") "
Oct 30 15:59:37.114233 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.114209    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-credential-socket\") pod \"a026ba91-cf65-4376-b0e2-386a4bd888ef\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") "
Oct 30 15:59:37.114362 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.114329    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-podinfo\") pod \"a026ba91-cf65-4376-b0e2-386a4bd888ef\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") "
Oct 30 15:59:37.114444 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.114419    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-data\") pod \"a026ba91-cf65-4376-b0e2-386a4bd888ef\" (UID: \"a026ba91-cf65-4376-b0e2-386a4bd888ef\") "
Oct 30 15:59:37.114893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.114163    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-socket" pod "a026ba91-cf65-4376-b0e2-386a4bd888ef" (UID: "a026ba91-cf65-4376-b0e2-386a4bd888ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.114893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.114310    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-certs" pod "a026ba91-cf65-4376-b0e2-386a4bd888ef" (UID: "a026ba91-cf65-4376-b0e2-386a4bd888ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.114893 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.114636    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a026ba91-cf65-4376-b0e2-386a4bd888ef-istiod-ca-cert" pod "a026ba91-cf65-4376-b0e2-386a4bd888ef" (UID: "a026ba91-cf65-4376-b0e2-386a4bd888ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:37.115181 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.115088    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-credential-socket" pod "a026ba91-cf65-4376-b0e2-386a4bd888ef" (UID: "a026ba91-cf65-4376-b0e2-386a4bd888ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.115247 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.115221    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.115307 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.115270    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.115307 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.115286    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a026ba91-cf65-4376-b0e2-386a4bd888ef-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.115307 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.115299    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.115504 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.115318    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-data" pod "a026ba91-cf65-4376-b0e2-386a4bd888ef" (UID: "a026ba91-cf65-4376-b0e2-386a4bd888ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.116626 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.116591    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-kube-api-access-kd87h" pod "a026ba91-cf65-4376-b0e2-386a4bd888ef" (UID: "a026ba91-cf65-4376-b0e2-386a4bd888ef"). InnerVolumeSpecName "kube-api-access-kd87h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:37.116770 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.116668    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-envoy" pod "a026ba91-cf65-4376-b0e2-386a4bd888ef" (UID: "a026ba91-cf65-4376-b0e2-386a4bd888ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.116916 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.116894    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-token" pod "a026ba91-cf65-4376-b0e2-386a4bd888ef" (UID: "a026ba91-cf65-4376-b0e2-386a4bd888ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:37.118943 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.118909    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-podinfo" pod "a026ba91-cf65-4376-b0e2-386a4bd888ef" (UID: "a026ba91-cf65-4376-b0e2-386a4bd888ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:59:37.215700 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.215664    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-data\") pod \"69f16429-e079-4aa7-adba-a8ceecd65b1f\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.215719    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-socket\") pod \"8e084541-731b-46dc-a2e0-dd989b1b36a6\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.215758    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-credential-socket\") pod \"8e084541-731b-46dc-a2e0-dd989b1b36a6\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.215798    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-socket\") pod \"69f16429-e079-4aa7-adba-a8ceecd65b1f\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.215841    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-envoy\") pod \"8e084541-731b-46dc-a2e0-dd989b1b36a6\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.215879    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-certs\") pod \"8e084541-731b-46dc-a2e0-dd989b1b36a6\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.215908    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-data\") pod \"8e084541-731b-46dc-a2e0-dd989b1b36a6\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.215942    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e084541-731b-46dc-a2e0-dd989b1b36a6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e084541-731b-46dc-a2e0-dd989b1b36a6-istiod-ca-cert\") pod \"8e084541-731b-46dc-a2e0-dd989b1b36a6\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.215971    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-token\" (UniqueName: \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-token\") pod \"8e084541-731b-46dc-a2e0-dd989b1b36a6\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216007    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-kube-api-access-vkjdk\" (UniqueName: \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-kube-api-access-vkjdk\") pod \"69f16429-e079-4aa7-adba-a8ceecd65b1f\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216034    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-podinfo\") pod \"8e084541-731b-46dc-a2e0-dd989b1b36a6\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") "
Oct 30 15:59:37.216087 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216076    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-kube-api-access-lsqr9\" (UniqueName: \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-kube-api-access-lsqr9\") pod \"8e084541-731b-46dc-a2e0-dd989b1b36a6\" (UID: \"8e084541-731b-46dc-a2e0-dd989b1b36a6\") "
Oct 30 15:59:37.216506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216115    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-certs\") pod \"69f16429-e079-4aa7-adba-a8ceecd65b1f\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") "
Oct 30 15:59:37.216506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216159    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-podinfo\") pod \"69f16429-e079-4aa7-adba-a8ceecd65b1f\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") "
Oct 30 15:59:37.216506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216188    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69f16429-e079-4aa7-adba-a8ceecd65b1f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f16429-e079-4aa7-adba-a8ceecd65b1f-istiod-ca-cert\") pod \"69f16429-e079-4aa7-adba-a8ceecd65b1f\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") "
Oct 30 15:59:37.216506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216299    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-credential-socket\") pod \"69f16429-e079-4aa7-adba-a8ceecd65b1f\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") "
Oct 30 15:59:37.216506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216392    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-envoy\") pod \"69f16429-e079-4aa7-adba-a8ceecd65b1f\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") "
Oct 30 15:59:37.216506 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216426    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-token\" (UniqueName: \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-token\") pod \"69f16429-e079-4aa7-adba-a8ceecd65b1f\" (UID: \"69f16429-e079-4aa7-adba-a8ceecd65b1f\") "
Oct 30 15:59:37.216911 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216891    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.216985 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216915    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.216985 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216928    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.216985 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216940    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.216985 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.216954    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kd87h\" (UniqueName: \"kubernetes.io/projected/a026ba91-cf65-4376-b0e2-386a4bd888ef-kube-api-access-kd87h\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.218422 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.218221    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-socket" pod "69f16429-e079-4aa7-adba-a8ceecd65b1f" (UID: "69f16429-e079-4aa7-adba-a8ceecd65b1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.218422 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.218296    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-certs" pod "69f16429-e079-4aa7-adba-a8ceecd65b1f" (UID: "69f16429-e079-4aa7-adba-a8ceecd65b1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.218808 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.218646    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-data" pod "69f16429-e079-4aa7-adba-a8ceecd65b1f" (UID: "69f16429-e079-4aa7-adba-a8ceecd65b1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.219121 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.218901    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-socket" pod "8e084541-731b-46dc-a2e0-dd989b1b36a6" (UID: "8e084541-731b-46dc-a2e0-dd989b1b36a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.219121 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.218920    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-credential-socket" pod "8e084541-731b-46dc-a2e0-dd989b1b36a6" (UID: "8e084541-731b-46dc-a2e0-dd989b1b36a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.219121 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.218929    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69f16429-e079-4aa7-adba-a8ceecd65b1f-istiod-ca-cert" pod "69f16429-e079-4aa7-adba-a8ceecd65b1f" (UID: "69f16429-e079-4aa7-adba-a8ceecd65b1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:37.219297 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.219150    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-credential-socket" pod "69f16429-e079-4aa7-adba-a8ceecd65b1f" (UID: "69f16429-e079-4aa7-adba-a8ceecd65b1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.219297 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.219207    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-certs" pod "8e084541-731b-46dc-a2e0-dd989b1b36a6" (UID: "8e084541-731b-46dc-a2e0-dd989b1b36a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.219549 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.219427    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-data" pod "8e084541-731b-46dc-a2e0-dd989b1b36a6" (UID: "8e084541-731b-46dc-a2e0-dd989b1b36a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.219725 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.219701    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e084541-731b-46dc-a2e0-dd989b1b36a6-istiod-ca-cert" pod "8e084541-731b-46dc-a2e0-dd989b1b36a6" (UID: "8e084541-731b-46dc-a2e0-dd989b1b36a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:37.220291 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.220262    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-token" pod "69f16429-e079-4aa7-adba-a8ceecd65b1f" (UID: "69f16429-e079-4aa7-adba-a8ceecd65b1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:37.220879 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.220849    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-podinfo" pod "8e084541-731b-46dc-a2e0-dd989b1b36a6" (UID: "8e084541-731b-46dc-a2e0-dd989b1b36a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:59:37.221457 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.221415    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-kube-api-access-lsqr9" pod "8e084541-731b-46dc-a2e0-dd989b1b36a6" (UID: "8e084541-731b-46dc-a2e0-dd989b1b36a6"). InnerVolumeSpecName "kube-api-access-lsqr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:37.221546 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.221473    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-token" pod "8e084541-731b-46dc-a2e0-dd989b1b36a6" (UID: "8e084541-731b-46dc-a2e0-dd989b1b36a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:37.222034 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.221989    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-podinfo" pod "69f16429-e079-4aa7-adba-a8ceecd65b1f" (UID: "69f16429-e079-4aa7-adba-a8ceecd65b1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 15:59:37.222034 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.221987    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-envoy" pod "69f16429-e079-4aa7-adba-a8ceecd65b1f" (UID: "69f16429-e079-4aa7-adba-a8ceecd65b1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.222471 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.222449    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-kube-api-access-vkjdk" pod "69f16429-e079-4aa7-adba-a8ceecd65b1f" (UID: "69f16429-e079-4aa7-adba-a8ceecd65b1f"). InnerVolumeSpecName "kube-api-access-vkjdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:37.222876 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.222860    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-envoy" pod "8e084541-731b-46dc-a2e0-dd989b1b36a6" (UID: "8e084541-731b-46dc-a2e0-dd989b1b36a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:37.318481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318398    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsqr9\" (UniqueName: \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-kube-api-access-lsqr9\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318426    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318437    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318446    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f16429-e079-4aa7-adba-a8ceecd65b1f-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318455    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318463    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318471    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318478    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318481 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318487    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318495    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318504    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f16429-e079-4aa7-adba-a8ceecd65b1f-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318514    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318522    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318529    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318536    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e084541-731b-46dc-a2e0-dd989b1b36a6-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318543    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318551    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkjdk\" (UniqueName: \"kubernetes.io/projected/69f16429-e079-4aa7-adba-a8ceecd65b1f-kube-api-access-vkjdk\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.318864 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.318558    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e084541-731b-46dc-a2e0-dd989b1b36a6-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:37.426282 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.426247    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"]
Oct 30 15:59:37.428581 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.428554    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-cn8qk"]
Oct 30 15:59:37.472320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.472285    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"]
Oct 30 15:59:37.477793 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:37.477764    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-r45lx"]
Oct 30 15:59:38.077289 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:38.077199    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"
Oct 30 15:59:38.077289 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:38.077207    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn" event={"ID":"8e084541-731b-46dc-a2e0-dd989b1b36a6","Type":"ContainerDied","Data":"6b0bb9518a50b5efd3ba0baf93499faa9026808e1a9abacac4caff4efa30dd9a"}
Oct 30 15:59:38.077521 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:38.077358    4585 scope.go:122] "RemoveContainer" containerID="888d9afa53ca835e0e432cb1662b5087e01fd6be46c3c4e07d9b58b6701ad720"
Oct 30 15:59:38.129636 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:38.129607    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"]
Oct 30 15:59:38.141510 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:38.141480    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-9e2b29-openshift-default-7cb44dddb7-xz8gn"]
Oct 30 15:59:38.701133 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:38.701093    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" path="/var/lib/kubelet/pods/69f16429-e079-4aa7-adba-a8ceecd65b1f/volumes"
Oct 30 15:59:38.701772 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:38.701751    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" path="/var/lib/kubelet/pods/8e084541-731b-46dc-a2e0-dd989b1b36a6/volumes"
Oct 30 15:59:38.702263 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:38.702242    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" path="/var/lib/kubelet/pods/a026ba91-cf65-4376-b0e2-386a4bd888ef/volumes"
Oct 30 15:59:39.938655 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:39.938617    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"]
Oct 30 15:59:39.984329 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:39.984292    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"]
Oct 30 15:59:47.116228 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116194    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"]
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116602    4585 cpu_manager.go:414] "containerMap: removing container" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116614    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116627    4585 cpu_manager.go:414] "containerMap: removing container" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116632    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116641    4585 cpu_manager.go:414] "containerMap: removing container" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116646    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116652    4585 cpu_manager.go:414] "containerMap: removing container" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116657    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116665    4585 cpu_manager.go:414] "containerMap: removing container" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116669    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116679    4585 cpu_manager.go:414] "containerMap: removing container" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116683    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116697    4585 cpu_manager.go:414] "containerMap: removing container" podUID="2598a7ca-70ff-4bd2-ae89-fa742412fa98" containerName="payload-pre-processing"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116702    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2598a7ca-70ff-4bd2-ae89-fa742412fa98" containerName="payload-pre-processing"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116708    4585 cpu_manager.go:414] "containerMap: removing container" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116713    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116724    4585 cpu_manager.go:414] "containerMap: removing container" podUID="547f6ffe-6f5a-4dac-8521-4e92df79f82e" containerName="payload-pre-processing"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116728    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="547f6ffe-6f5a-4dac-8521-4e92df79f82e" containerName="payload-pre-processing"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116734    4585 cpu_manager.go:414] "containerMap: removing container" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy"
Oct 30 15:59:47.116732 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116739    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy"
Oct 30 15:59:47.117318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116797    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="9400972d-230d-4521-a683-bfc605f3b647" containerName="istio-proxy"
Oct 30 15:59:47.117318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116804    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a0bee82-0d1d-4bb9-8575-3683fe99d5f2" containerName="istio-proxy"
Oct 30 15:59:47.117318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116810    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="4db0f08f-678a-4f7e-99ee-8c7bb4de5af5" containerName="istio-proxy"
Oct 30 15:59:47.117318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116817    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e084541-731b-46dc-a2e0-dd989b1b36a6" containerName="istio-proxy"
Oct 30 15:59:47.117318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116822    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd1d3da4-a476-4f37-b2c6-acc545bfd734" containerName="istio-proxy"
Oct 30 15:59:47.117318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116828    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc27ad1e-45a0-4a8f-bec8-ec1e6a6468eb" containerName="istio-proxy"
Oct 30 15:59:47.117318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116833    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="a026ba91-cf65-4376-b0e2-386a4bd888ef" containerName="istio-proxy"
Oct 30 15:59:47.117318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116841    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="2598a7ca-70ff-4bd2-ae89-fa742412fa98" containerName="payload-pre-processing"
Oct 30 15:59:47.117318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116846    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="69f16429-e079-4aa7-adba-a8ceecd65b1f" containerName="istio-proxy"
Oct 30 15:59:47.117318 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.116851    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="547f6ffe-6f5a-4dac-8521-4e92df79f82e" containerName="payload-pre-processing"
Oct 30 15:59:47.121773 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.121751    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.129038 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.129009    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc71e045-882pl\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.129167 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.129052    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc71e045-882pl\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.129320 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.129291    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ql77\" (UniqueName: \"kubernetes.io/projected/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-kube-api-access-6ql77\") pod \"maas-api-revoke-sub-cc71e045-882pl\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.146229 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.146197    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"]
Oct 30 15:59:47.214011 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.213975    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 15:59:47.214193 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.214025    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 15:59:47.217077 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.217054    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 15:59:47.217171 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.217055    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 15:59:47.230719 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.230689    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc71e045-882pl\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.230805 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.230725    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc71e045-882pl\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.230845 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.230821    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ql77\" (UniqueName: \"kubernetes.io/projected/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-kube-api-access-6ql77\") pod \"maas-api-revoke-sub-cc71e045-882pl\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.231540 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.231520    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc71e045-882pl\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.233173 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.233155    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc71e045-882pl\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.254658 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.254628    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ql77\" (UniqueName: \"kubernetes.io/projected/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-kube-api-access-6ql77\") pod \"maas-api-revoke-sub-cc71e045-882pl\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.436287 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.436243    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:47.762305 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:47.762281    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"]
Oct 30 15:59:47.762906 ip-10-0-140-110 kubenswrapper[4585]: W1030 15:59:47.762881    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f21b8b8_8a1a_4bdc_bfb7_4c96c09cb0ee.slice/crio-731aea90a09175966b3333f77f5d26db3ab8353bf527e7850cfd2b6f60865e3e WatchSource:0}: Error finding container 731aea90a09175966b3333f77f5d26db3ab8353bf527e7850cfd2b6f60865e3e: Status 404 returned error can't find the container with id 731aea90a09175966b3333f77f5d26db3ab8353bf527e7850cfd2b6f60865e3e
Oct 30 15:59:47.964993 ip-10-0-140-110 kubenswrapper[4585]: E1030 15:59:47.964948    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 15:59:48.135966 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:48.135926    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl" event={"ID":"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee","Type":"ContainerStarted","Data":"9a3d33dd16afeb17e96b49b771f3fafabba578405051f8b58e596950776521a1"}
Oct 30 15:59:48.136393 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:48.135982    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl" event={"ID":"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee","Type":"ContainerStarted","Data":"731aea90a09175966b3333f77f5d26db3ab8353bf527e7850cfd2b6f60865e3e"}
Oct 30 15:59:48.181828 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:48.181778    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl" podStartSLOduration=1.181763449 podStartE2EDuration="1.181763449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 15:59:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 15:59:48.18120612 +0000 UTC m=+1297.616810468" watchObservedRunningTime="2026-10-30 15:59:48.181763449 +0000 UTC m=+1297.617367796"
Oct 30 15:59:49.142672 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:49.142641    4585 generic.go:356] "Generic (PLEG): container finished" podID="1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee" containerID="9a3d33dd16afeb17e96b49b771f3fafabba578405051f8b58e596950776521a1" exitCode=0
Oct 30 15:59:49.143072 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:49.142729    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl" event={"ID":"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee","Type":"ContainerDied","Data":"9a3d33dd16afeb17e96b49b771f3fafabba578405051f8b58e596950776521a1"}
Oct 30 15:59:50.294899 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:50.294870    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:50.366022 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:50.365988    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-kube-api-access-6ql77\" (UniqueName: \"kubernetes.io/projected/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-kube-api-access-6ql77\") pod \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") "
Oct 30 15:59:50.366201 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:50.366110    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-xks-ca\") pod \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") "
Oct 30 15:59:50.366272 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:50.366221    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-openshift-ca\") pod \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\" (UID: \"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee\") "
Oct 30 15:59:50.366676 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:50.366644    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-openshift-ca" pod "1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee" (UID: "1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 15:59:50.368278 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:50.368245    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-xks-ca" pod "1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee" (UID: "1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:59:50.368396 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:50.368250    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-kube-api-access-6ql77" pod "1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee" (UID: "1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee"). InnerVolumeSpecName "kube-api-access-6ql77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:50.467322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:50.467228    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ql77\" (UniqueName: \"kubernetes.io/projected/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-kube-api-access-6ql77\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:50.467322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:50.467263    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:50.467322 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:50.467273    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:51.153800 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:51.153769    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"
Oct 30 15:59:51.153992 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:51.153804    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl" event={"ID":"1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee","Type":"ContainerDied","Data":"731aea90a09175966b3333f77f5d26db3ab8353bf527e7850cfd2b6f60865e3e"}
Oct 30 15:59:51.153992 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:51.153854    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="731aea90a09175966b3333f77f5d26db3ab8353bf527e7850cfd2b6f60865e3e"
Oct 30 15:59:54.958408 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:54.958311    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" containerID="cri-o://f0c4621445ec8e12b12a9a24d7622193a17d8dc075ad19ede1439a6cf7646c03" gracePeriod=45
Oct 30 15:59:55.004183 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.004136    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" containerID="cri-o://c94338bb56283fa7176ab8799a224dbe42c898ce2eedf8a3f14ffa4a857ed05d" gracePeriod=45
Oct 30 15:59:55.037338 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.037305    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": dial tcp 10.134.0.55:8000: connect: connection refused"
Oct 30 15:59:55.056966 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.056925    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": dial tcp 10.134.0.72:8000: connect: connection refused"
Oct 30 15:59:55.179268 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.179237    4585 generic.go:356] "Generic (PLEG): container finished" podID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerID="f0c4621445ec8e12b12a9a24d7622193a17d8dc075ad19ede1439a6cf7646c03" exitCode=0
Oct 30 15:59:55.179462 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.179321    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" event={"ID":"c1a469f1-e1a1-4800-8827-14e7ce0501cc","Type":"ContainerDied","Data":"f0c4621445ec8e12b12a9a24d7622193a17d8dc075ad19ede1439a6cf7646c03"}
Oct 30 15:59:55.181123 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.181103    4585 generic.go:356] "Generic (PLEG): container finished" podID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerID="c94338bb56283fa7176ab8799a224dbe42c898ce2eedf8a3f14ffa4a857ed05d" exitCode=0
Oct 30 15:59:55.181228 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.181170    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" event={"ID":"fb180805-8d51-47d0-8b08-4c26e39c8c50","Type":"ContainerDied","Data":"c94338bb56283fa7176ab8799a224dbe42c898ce2eedf8a3f14ffa4a857ed05d"}
Oct 30 15:59:55.278772 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.278627    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:59:55.301209 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.301184    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:59:55.318337 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.318312    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tls-certs\" (UniqueName: \"kubernetes.io/secret/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tls-certs\") pod \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") "
Oct 30 15:59:55.318496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.318444    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-model-cache\") pod \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") "
Oct 30 15:59:55.318496 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.318485    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-dshm\") pod \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") "
Oct 30 15:59:55.318616 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.318556    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1a469f1-e1a1-4800-8827-14e7ce0501cc-kube-api-access-6dpqs\" (UniqueName: \"kubernetes.io/projected/c1a469f1-e1a1-4800-8827-14e7ce0501cc-kube-api-access-6dpqs\") pod \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") "
Oct 30 15:59:55.318670 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.318614    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tmp-dir\") pod \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") "
Oct 30 15:59:55.318736 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.318723    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-home\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-home\") pod \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\" (UID: \"c1a469f1-e1a1-4800-8827-14e7ce0501cc\") "
Oct 30 15:59:55.318944 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.318717    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-model-cache" pod "c1a469f1-e1a1-4800-8827-14e7ce0501cc" (UID: "c1a469f1-e1a1-4800-8827-14e7ce0501cc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:55.319023 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.318871    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tmp-dir" pod "c1a469f1-e1a1-4800-8827-14e7ce0501cc" (UID: "c1a469f1-e1a1-4800-8827-14e7ce0501cc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:55.319023 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.318981    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-home" pod "c1a469f1-e1a1-4800-8827-14e7ce0501cc" (UID: "c1a469f1-e1a1-4800-8827-14e7ce0501cc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:55.319172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.319154    4585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tmp-dir\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.319172 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.319179    4585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-home\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.319264 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.319192    4585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-model-cache\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.320926 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.320752    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tls-certs" pod "c1a469f1-e1a1-4800-8827-14e7ce0501cc" (UID: "c1a469f1-e1a1-4800-8827-14e7ce0501cc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:59:55.321596 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.321433    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-dshm" pod "c1a469f1-e1a1-4800-8827-14e7ce0501cc" (UID: "c1a469f1-e1a1-4800-8827-14e7ce0501cc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:55.321596 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.321521    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1a469f1-e1a1-4800-8827-14e7ce0501cc-kube-api-access-6dpqs" pod "c1a469f1-e1a1-4800-8827-14e7ce0501cc" (UID: "c1a469f1-e1a1-4800-8827-14e7ce0501cc"). InnerVolumeSpecName "kube-api-access-6dpqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:55.420246 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.420206    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb180805-8d51-47d0-8b08-4c26e39c8c50-kube-api-access-plqnb\" (UniqueName: \"kubernetes.io/projected/fb180805-8d51-47d0-8b08-4c26e39c8c50-kube-api-access-plqnb\") pod \"fb180805-8d51-47d0-8b08-4c26e39c8c50\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") "
Oct 30 15:59:55.420463 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.420267    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-model-cache\") pod \"fb180805-8d51-47d0-8b08-4c26e39c8c50\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") "
Oct 30 15:59:55.420463 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.420291    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-dshm\") pod \"fb180805-8d51-47d0-8b08-4c26e39c8c50\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") "
Oct 30 15:59:55.420463 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.420325    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb180805-8d51-47d0-8b08-4c26e39c8c50-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb180805-8d51-47d0-8b08-4c26e39c8c50-tls-certs\") pod \"fb180805-8d51-47d0-8b08-4c26e39c8c50\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") "
Oct 30 15:59:55.420647 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.420470    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-tmp-dir\") pod \"fb180805-8d51-47d0-8b08-4c26e39c8c50\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") "
Oct 30 15:59:55.420647 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.420536    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-home\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-home\") pod \"fb180805-8d51-47d0-8b08-4c26e39c8c50\" (UID: \"fb180805-8d51-47d0-8b08-4c26e39c8c50\") "
Oct 30 15:59:55.420750 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.420629    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-model-cache" pod "fb180805-8d51-47d0-8b08-4c26e39c8c50" (UID: "fb180805-8d51-47d0-8b08-4c26e39c8c50"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:55.420829 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.420794    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-home" pod "fb180805-8d51-47d0-8b08-4c26e39c8c50" (UID: "fb180805-8d51-47d0-8b08-4c26e39c8c50"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:55.421028 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.421002    4585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-home\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.421028 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.421030    4585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1a469f1-e1a1-4800-8827-14e7ce0501cc-dshm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.421244 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.421044    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dpqs\" (UniqueName: \"kubernetes.io/projected/c1a469f1-e1a1-4800-8827-14e7ce0501cc-kube-api-access-6dpqs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.421244 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.421060    4585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-model-cache\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.421244 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.421074    4585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1a469f1-e1a1-4800-8827-14e7ce0501cc-tls-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.421439 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.421272    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-tmp-dir" pod "fb180805-8d51-47d0-8b08-4c26e39c8c50" (UID: "fb180805-8d51-47d0-8b08-4c26e39c8c50"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:55.422556 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.422522    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-dshm" pod "fb180805-8d51-47d0-8b08-4c26e39c8c50" (UID: "fb180805-8d51-47d0-8b08-4c26e39c8c50"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 15:59:55.422675 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.422601    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb180805-8d51-47d0-8b08-4c26e39c8c50-kube-api-access-plqnb" pod "fb180805-8d51-47d0-8b08-4c26e39c8c50" (UID: "fb180805-8d51-47d0-8b08-4c26e39c8c50"). InnerVolumeSpecName "kube-api-access-plqnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 15:59:55.422675 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.422604    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb180805-8d51-47d0-8b08-4c26e39c8c50-tls-certs" pod "fb180805-8d51-47d0-8b08-4c26e39c8c50" (UID: "fb180805-8d51-47d0-8b08-4c26e39c8c50"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 15:59:55.522641 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.522551    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plqnb\" (UniqueName: \"kubernetes.io/projected/fb180805-8d51-47d0-8b08-4c26e39c8c50-kube-api-access-plqnb\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.522641 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.522584    4585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-dshm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.522641 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.522595    4585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb180805-8d51-47d0-8b08-4c26e39c8c50-tls-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:55.522641 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:55.522604    4585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb180805-8d51-47d0-8b08-4c26e39c8c50-tmp-dir\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 15:59:56.187842 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.187805    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"
Oct 30 15:59:56.187842 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.187835    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d" event={"ID":"c1a469f1-e1a1-4800-8827-14e7ce0501cc","Type":"ContainerDied","Data":"063b40b252660a64f23c251cc1d9ce5ff173496a52fea10290b0277e76be7b03"}
Oct 30 15:59:56.188378 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.187894    4585 scope.go:122] "RemoveContainer" containerID="f0c4621445ec8e12b12a9a24d7622193a17d8dc075ad19ede1439a6cf7646c03"
Oct 30 15:59:56.189256 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.189237    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"
Oct 30 15:59:56.189337 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.189235    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758" event={"ID":"fb180805-8d51-47d0-8b08-4c26e39c8c50","Type":"ContainerDied","Data":"8b130415345ab2cf622ea9461f2236983511a1b3b59e562d2d6e286ec836eb2a"}
Oct 30 15:59:56.197957 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.197937    4585 scope.go:122] "RemoveContainer" containerID="c94338bb56283fa7176ab8799a224dbe42c898ce2eedf8a3f14ffa4a857ed05d"
Oct 30 15:59:56.238262 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.238173    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"]
Oct 30 15:59:56.250211 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.250181    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/premium-w5-ba9db9-kserve-9dc89d6f8-km758"]
Oct 30 15:59:56.283610 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.283577    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"]
Oct 30 15:59:56.287783 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.287754    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-ba9db9/distinct-w5-ba9db9-kserve-66556578b5-bgv9d"]
Oct 30 15:59:56.700091 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.700063    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" path="/var/lib/kubelet/pods/c1a469f1-e1a1-4800-8827-14e7ce0501cc/volumes"
Oct 30 15:59:56.700488 ip-10-0-140-110 kubenswrapper[4585]: I1030 15:59:56.700474    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" path="/var/lib/kubelet/pods/fb180805-8d51-47d0-8b08-4c26e39c8c50/volumes"
Oct 30 16:00:00.161961 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.161926    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"]
Oct 30 16:00:00.162356 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.162296    4585 cpu_manager.go:414] "containerMap: removing container" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main"
Oct 30 16:00:00.162356 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.162307    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main"
Oct 30 16:00:00.162356 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.162321    4585 cpu_manager.go:414] "containerMap: removing container" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main"
Oct 30 16:00:00.162356 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.162326    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main"
Oct 30 16:00:00.162494 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.162363    4585 cpu_manager.go:414] "containerMap: removing container" podUID="1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee" containerName="revoke-keys"
Oct 30 16:00:00.162494 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.162369    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee" containerName="revoke-keys"
Oct 30 16:00:00.162494 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.162423    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee" containerName="revoke-keys"
Oct 30 16:00:00.162494 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.162432    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1a469f1-e1a1-4800-8827-14e7ce0501cc" containerName="main"
Oct 30 16:00:00.162494 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.162439    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb180805-8d51-47d0-8b08-4c26e39c8c50" containerName="main"
Oct 30 16:00:00.165582 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.165554    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"
Oct 30 16:00:00.187257 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.187225    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"]
Oct 30 16:00:00.196036 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.196001    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 16:00:00.196163 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.196007    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 16:00:00.196163 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.196094    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 16:00:00.269480 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.269443    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889600-np7nn\" (UID: \"366f8b79-c76d-4d7c-a94f-95f5e4e4be26\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"
Oct 30 16:00:00.269670 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.269617    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr9g8\" (UniqueName: \"kubernetes.io/projected/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-kube-api-access-zr9g8\") pod \"maas-api-key-cleanup-29889600-np7nn\" (UID: \"366f8b79-c76d-4d7c-a94f-95f5e4e4be26\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"
Oct 30 16:00:00.370343 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.370301    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr9g8\" (UniqueName: \"kubernetes.io/projected/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-kube-api-access-zr9g8\") pod \"maas-api-key-cleanup-29889600-np7nn\" (UID: \"366f8b79-c76d-4d7c-a94f-95f5e4e4be26\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"
Oct 30 16:00:00.370538 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.370445    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889600-np7nn\" (UID: \"366f8b79-c76d-4d7c-a94f-95f5e4e4be26\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"
Oct 30 16:00:00.371116 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.371095    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889600-np7nn\" (UID: \"366f8b79-c76d-4d7c-a94f-95f5e4e4be26\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"
Oct 30 16:00:00.394012 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.393968    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr9g8\" (UniqueName: \"kubernetes.io/projected/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-kube-api-access-zr9g8\") pod \"maas-api-key-cleanup-29889600-np7nn\" (UID: \"366f8b79-c76d-4d7c-a94f-95f5e4e4be26\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"
Oct 30 16:00:00.478608 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.478524    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"
Oct 30 16:00:00.794229 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:00.794198    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"]
Oct 30 16:00:00.796569 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:00:00.796535    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod366f8b79_c76d_4d7c_a94f_95f5e4e4be26.slice/crio-7047dde17046622d33824f47ce640133f4ade6b0999aa764a47a2c9e9f3fa501 WatchSource:0}: Error finding container 7047dde17046622d33824f47ce640133f4ade6b0999aa764a47a2c9e9f3fa501: Status 404 returned error can't find the container with id 7047dde17046622d33824f47ce640133f4ade6b0999aa764a47a2c9e9f3fa501
Oct 30 16:00:01.224049 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:01.224004    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" event={"ID":"366f8b79-c76d-4d7c-a94f-95f5e4e4be26","Type":"ContainerStarted","Data":"0fc5a7faaf98742dddd2e7f18a2118b81b5847ac80cf75898f193b013003683f"}
Oct 30 16:00:01.224612 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:01.224080    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" event={"ID":"366f8b79-c76d-4d7c-a94f-95f5e4e4be26","Type":"ContainerStarted","Data":"7047dde17046622d33824f47ce640133f4ade6b0999aa764a47a2c9e9f3fa501"}
Oct 30 16:00:01.267053 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:01.266991    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" podStartSLOduration=1.266973003 podStartE2EDuration="1.266973003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:00:01.265809817 +0000 UTC m=+1310.701414170" watchObservedRunningTime="2026-10-30 16:00:01.266973003 +0000 UTC m=+1310.702577355"
Oct 30 16:00:02.271378 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:02.271224    4585 generic.go:356] "Generic (PLEG): container finished" podID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerID="0fc5a7faaf98742dddd2e7f18a2118b81b5847ac80cf75898f193b013003683f" exitCode=60
Oct 30 16:00:02.271378 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:02.271337    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" event={"ID":"366f8b79-c76d-4d7c-a94f-95f5e4e4be26","Type":"ContainerDied","Data":"0fc5a7faaf98742dddd2e7f18a2118b81b5847ac80cf75898f193b013003683f"}
Oct 30 16:00:02.271972 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:02.271779    4585 scope.go:122] "RemoveContainer" containerID="0fc5a7faaf98742dddd2e7f18a2118b81b5847ac80cf75898f193b013003683f"
Oct 30 16:00:03.278231 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:03.278199    4585 generic.go:356] "Generic (PLEG): container finished" podID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerID="0b9d846721b78c547a924dcc9a2b1d31d312c317f8f612b08020f60675a60c6d" exitCode=60
Oct 30 16:00:03.278695 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:03.278284    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" event={"ID":"366f8b79-c76d-4d7c-a94f-95f5e4e4be26","Type":"ContainerDied","Data":"0b9d846721b78c547a924dcc9a2b1d31d312c317f8f612b08020f60675a60c6d"}
Oct 30 16:00:03.278695 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:03.278340    4585 scope.go:122] "RemoveContainer" containerID="0fc5a7faaf98742dddd2e7f18a2118b81b5847ac80cf75898f193b013003683f"
Oct 30 16:00:03.278695 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:03.278627    4585 scope.go:122] "RemoveContainer" containerID="0b9d846721b78c547a924dcc9a2b1d31d312c317f8f612b08020f60675a60c6d"
Oct 30 16:00:03.278944 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:00:03.278918    4585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889600-np7nn_odh-ai-gateway-infra(366f8b79-c76d-4d7c-a94f-95f5e4e4be26)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26"
Oct 30 16:00:04.286214 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:04.286186    4585 scope.go:122] "RemoveContainer" containerID="0b9d846721b78c547a924dcc9a2b1d31d312c317f8f612b08020f60675a60c6d"
Oct 30 16:00:04.286645 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:00:04.286462    4585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889600-np7nn_odh-ai-gateway-infra(366f8b79-c76d-4d7c-a94f-95f5e4e4be26)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26"
Oct 30 16:00:10.161050 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:10.161007    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"]
Oct 30 16:00:10.161513 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:10.161289    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" containerID="cri-o://d9548d16c12bcfc4480db3c701fc64569c8926c4803e2384a4f7409ac110cbc5" gracePeriod=30
Oct 30 16:00:10.173072 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:10.173037    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"]
Oct 30 16:00:10.173457 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:10.173422    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy" containerID="cri-o://cf199c732f2ba03c1cbc0781f88a36353c7972bb86030c70603749dcabaf97dd" gracePeriod=30
Oct 30 16:00:10.177879 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:10.177852    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"]
Oct 30 16:00:10.178102 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:10.178057    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" containerID="cri-o://adcdafd246cf0ec79c776b1097f171f65a8a9efb60beb5eb68fc8b725cbf7a96" gracePeriod=30
Oct 30 16:00:11.755556 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:11.755516    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 16:00:11.756073 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:11.755609    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 16:00:11.785063 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:11.785031    4585 scope.go:122] "RemoveContainer" containerID="466c1143d3ca9ce8ba52f6acc78f845f8d53bb011cfb06f1eb7576405e0420f1"
Oct 30 16:00:11.967177 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:11.967132    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 16:00:11.967404 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:11.967216    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 16:00:15.293663 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:00:15.293617    4585 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50305fbb_13d9_4263_86f9_6ed6b5adf9b3.slice/crio-adcdafd246cf0ec79c776b1097f171f65a8a9efb60beb5eb68fc8b725cbf7a96.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809eb8ab_fb39_471e_a770_3eaea55689a9.slice/crio-conmon-d9548d16c12bcfc4480db3c701fc64569c8926c4803e2384a4f7409ac110cbc5.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50305fbb_13d9_4263_86f9_6ed6b5adf9b3.slice/crio-conmon-adcdafd246cf0ec79c776b1097f171f65a8a9efb60beb5eb68fc8b725cbf7a96.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809eb8ab_fb39_471e_a770_3eaea55689a9.slice/crio-d9548d16c12bcfc4480db3c701fc64569c8926c4803e2384a4f7409ac110cbc5.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod876862ba_27c9_4c8b_b440_93ec2241b948.slice/crio-conmon-cf199c732f2ba03c1cbc0781f88a36353c7972bb86030c70603749dcabaf97dd.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 16:00:15.294060 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:00:15.293769    4585 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50305fbb_13d9_4263_86f9_6ed6b5adf9b3.slice/crio-adcdafd246cf0ec79c776b1097f171f65a8a9efb60beb5eb68fc8b725cbf7a96.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809eb8ab_fb39_471e_a770_3eaea55689a9.slice/crio-conmon-d9548d16c12bcfc4480db3c701fc64569c8926c4803e2384a4f7409ac110cbc5.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50305fbb_13d9_4263_86f9_6ed6b5adf9b3.slice/crio-conmon-adcdafd246cf0ec79c776b1097f171f65a8a9efb60beb5eb68fc8b725cbf7a96.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809eb8ab_fb39_471e_a770_3eaea55689a9.slice/crio-d9548d16c12bcfc4480db3c701fc64569c8926c4803e2384a4f7409ac110cbc5.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809eb8ab_fb39_471e_a770_3eaea55689a9.slice/crio-a059045dcf701083e11ce48558eb60086301b654e9a9d2bf0d16d8684e1c10ad\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod876862ba_27c9_4c8b_b440_93ec2241b948.slice/crio-conmon-cf199c732f2ba03c1cbc0781f88a36353c7972bb86030c70603749dcabaf97dd.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 16:00:15.354066 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.353497    4585 generic.go:356] "Generic (PLEG): container finished" podID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerID="adcdafd246cf0ec79c776b1097f171f65a8a9efb60beb5eb68fc8b725cbf7a96" exitCode=0
Oct 30 16:00:15.354066 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.353590    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" event={"ID":"50305fbb-13d9-4263-86f9-6ed6b5adf9b3","Type":"ContainerDied","Data":"adcdafd246cf0ec79c776b1097f171f65a8a9efb60beb5eb68fc8b725cbf7a96"}
Oct 30 16:00:15.356458 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.356179    4585 generic.go:356] "Generic (PLEG): container finished" podID="876862ba-27c9-4c8b-b440-93ec2241b948" containerID="cf199c732f2ba03c1cbc0781f88a36353c7972bb86030c70603749dcabaf97dd" exitCode=0
Oct 30 16:00:15.356458 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.356276    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" event={"ID":"876862ba-27c9-4c8b-b440-93ec2241b948","Type":"ContainerDied","Data":"cf199c732f2ba03c1cbc0781f88a36353c7972bb86030c70603749dcabaf97dd"}
Oct 30 16:00:15.356458 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.356328    4585 scope.go:122] "RemoveContainer" containerID="8a8dc2bc4742a8d962906026407881e10956fca1de9c7aeb341879a4dca5f88c"
Oct 30 16:00:15.360918 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.360840    4585 generic.go:356] "Generic (PLEG): container finished" podID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerID="d9548d16c12bcfc4480db3c701fc64569c8926c4803e2384a4f7409ac110cbc5" exitCode=0
Oct 30 16:00:15.361193 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.361065    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" event={"ID":"809eb8ab-fb39-471e-a770-3eaea55689a9","Type":"ContainerDied","Data":"d9548d16c12bcfc4480db3c701fc64569c8926c4803e2384a4f7409ac110cbc5"}
Oct 30 16:00:15.479781 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.479464    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 16:00:15.498212 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.498184    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 16:00:15.561830 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.561782    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-certs\") pod \"876862ba-27c9-4c8b-b440-93ec2241b948\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") "
Oct 30 16:00:15.562036 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.561855    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-envoy\") pod \"809eb8ab-fb39-471e-a770-3eaea55689a9\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") "
Oct 30 16:00:15.562036 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.561891    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/876862ba-27c9-4c8b-b440-93ec2241b948-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/876862ba-27c9-4c8b-b440-93ec2241b948-istio-podinfo\") pod \"876862ba-27c9-4c8b-b440-93ec2241b948\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") "
Oct 30 16:00:15.562036 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.561941    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-istio-token\" (UniqueName: \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-istio-token\") pod \"876862ba-27c9-4c8b-b440-93ec2241b948\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") "
Oct 30 16:00:15.562036 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.561975    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/809eb8ab-fb39-471e-a770-3eaea55689a9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/809eb8ab-fb39-471e-a770-3eaea55689a9-istiod-ca-cert\") pod \"809eb8ab-fb39-471e-a770-3eaea55689a9\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") "
Oct 30 16:00:15.562036 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562033    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-credential-socket\") pod \"809eb8ab-fb39-471e-a770-3eaea55689a9\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") "
Oct 30 16:00:15.562337 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562069    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/876862ba-27c9-4c8b-b440-93ec2241b948-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/876862ba-27c9-4c8b-b440-93ec2241b948-istiod-ca-cert\") pod \"876862ba-27c9-4c8b-b440-93ec2241b948\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") "
Oct 30 16:00:15.562337 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562117    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-certs\") pod \"809eb8ab-fb39-471e-a770-3eaea55689a9\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") "
Oct 30 16:00:15.562337 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562154    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-data\") pod \"809eb8ab-fb39-471e-a770-3eaea55689a9\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") "
Oct 30 16:00:15.562337 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562190    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-socket\") pod \"876862ba-27c9-4c8b-b440-93ec2241b948\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") "
Oct 30 16:00:15.562337 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562264    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-podinfo\") pod \"809eb8ab-fb39-471e-a770-3eaea55689a9\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") "
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562404    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-socket\") pod \"809eb8ab-fb39-471e-a770-3eaea55689a9\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") "
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562477    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-kube-api-access-8mhnv\" (UniqueName: \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-kube-api-access-8mhnv\") pod \"876862ba-27c9-4c8b-b440-93ec2241b948\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") "
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562557    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-envoy\") pod \"876862ba-27c9-4c8b-b440-93ec2241b948\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") "
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562602    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-credential-socket\") pod \"876862ba-27c9-4c8b-b440-93ec2241b948\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") "
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562636    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-kube-api-access-l6d9x\" (UniqueName: \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-kube-api-access-l6d9x\") pod \"809eb8ab-fb39-471e-a770-3eaea55689a9\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") "
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562707    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-data\") pod \"876862ba-27c9-4c8b-b440-93ec2241b948\" (UID: \"876862ba-27c9-4c8b-b440-93ec2241b948\") "
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.562736    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-token\" (UniqueName: \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-token\") pod \"809eb8ab-fb39-471e-a770-3eaea55689a9\" (UID: \"809eb8ab-fb39-471e-a770-3eaea55689a9\") "
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.563212    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-data" pod "809eb8ab-fb39-471e-a770-3eaea55689a9" (UID: "809eb8ab-fb39-471e-a770-3eaea55689a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.563515    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-socket" pod "876862ba-27c9-4c8b-b440-93ec2241b948" (UID: "876862ba-27c9-4c8b-b440-93ec2241b948"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.563526    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/876862ba-27c9-4c8b-b440-93ec2241b948-istiod-ca-cert" pod "876862ba-27c9-4c8b-b440-93ec2241b948" (UID: "876862ba-27c9-4c8b-b440-93ec2241b948"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.563911    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/809eb8ab-fb39-471e-a770-3eaea55689a9-istiod-ca-cert" pod "809eb8ab-fb39-471e-a770-3eaea55689a9" (UID: "809eb8ab-fb39-471e-a770-3eaea55689a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.564233    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-credential-socket" pod "809eb8ab-fb39-471e-a770-3eaea55689a9" (UID: "809eb8ab-fb39-471e-a770-3eaea55689a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.564411    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-certs" pod "809eb8ab-fb39-471e-a770-3eaea55689a9" (UID: "809eb8ab-fb39-471e-a770-3eaea55689a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.565053    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-credential-socket" pod "876862ba-27c9-4c8b-b440-93ec2241b948" (UID: "876862ba-27c9-4c8b-b440-93ec2241b948"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.565386    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-data" pod "876862ba-27c9-4c8b-b440-93ec2241b948" (UID: "876862ba-27c9-4c8b-b440-93ec2241b948"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.566103    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-certs" pod "876862ba-27c9-4c8b-b440-93ec2241b948" (UID: "876862ba-27c9-4c8b-b440-93ec2241b948"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.568673 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.566415    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-socket" pod "809eb8ab-fb39-471e-a770-3eaea55689a9" (UID: "809eb8ab-fb39-471e-a770-3eaea55689a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.582570 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.582411    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/876862ba-27c9-4c8b-b440-93ec2241b948-istio-podinfo" pod "876862ba-27c9-4c8b-b440-93ec2241b948" (UID: "876862ba-27c9-4c8b-b440-93ec2241b948"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 16:00:15.583936 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.583887    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-token" pod "809eb8ab-fb39-471e-a770-3eaea55689a9" (UID: "809eb8ab-fb39-471e-a770-3eaea55689a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:00:15.584062 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.583969    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-istio-token" pod "876862ba-27c9-4c8b-b440-93ec2241b948" (UID: "876862ba-27c9-4c8b-b440-93ec2241b948"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:00:15.584248 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.584217    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-envoy" pod "809eb8ab-fb39-471e-a770-3eaea55689a9" (UID: "809eb8ab-fb39-471e-a770-3eaea55689a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.584507 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.584479    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-kube-api-access-8mhnv" pod "876862ba-27c9-4c8b-b440-93ec2241b948" (UID: "876862ba-27c9-4c8b-b440-93ec2241b948"). InnerVolumeSpecName "kube-api-access-8mhnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:00:15.584588 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.584522    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-kube-api-access-l6d9x" pod "809eb8ab-fb39-471e-a770-3eaea55689a9" (UID: "809eb8ab-fb39-471e-a770-3eaea55689a9"). InnerVolumeSpecName "kube-api-access-l6d9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:00:15.586503 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.586480    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-podinfo" pod "809eb8ab-fb39-471e-a770-3eaea55689a9" (UID: "809eb8ab-fb39-471e-a770-3eaea55689a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 16:00:15.587212 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.587180    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-envoy" pod "876862ba-27c9-4c8b-b440-93ec2241b948" (UID: "876862ba-27c9-4c8b-b440-93ec2241b948"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.605986 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.599680    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 16:00:15.664487 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.664441    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-certs\") pod \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") "
Oct 30 16:00:15.664658 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.664512    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-data\") pod \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") "
Oct 30 16:00:15.664658 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.664557    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istiod-ca-cert\") pod \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") "
Oct 30 16:00:15.664658 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.664603    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-token\" (UniqueName: \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-token\") pod \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") "
Oct 30 16:00:15.664658 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.664634    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-credential-socket\") pod \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") "
Oct 30 16:00:15.664909 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.664669    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-podinfo\") pod \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") "
Oct 30 16:00:15.664909 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.664756    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-kube-api-access-m48vq\" (UniqueName: \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-kube-api-access-m48vq\") pod \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") "
Oct 30 16:00:15.664909 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.664865    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-socket\") pod \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") "
Oct 30 16:00:15.665076 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.664926    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-envoy\") pod \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\" (UID: \"50305fbb-13d9-4263-86f9-6ed6b5adf9b3\") "
Oct 30 16:00:15.665338 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665312    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665338 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665359    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/809eb8ab-fb39-471e-a770-3eaea55689a9-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665376    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665392    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/876862ba-27c9-4c8b-b440-93ec2241b948-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665406    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665419    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665432    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665445    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665459    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665476    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mhnv\" (UniqueName: \"kubernetes.io/projected/876862ba-27c9-4c8b-b440-93ec2241b948-kube-api-access-8mhnv\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665489    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665503    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665517    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6d9x\" (UniqueName: \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-kube-api-access-l6d9x\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.665530 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665531    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.666159 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665584    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.666159 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665600    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/876862ba-27c9-4c8b-b440-93ec2241b948-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.666159 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665613    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/809eb8ab-fb39-471e-a770-3eaea55689a9-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.666159 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.665627    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/876862ba-27c9-4c8b-b440-93ec2241b948-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.666960 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.666915    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-data" pod "50305fbb-13d9-4263-86f9-6ed6b5adf9b3" (UID: "50305fbb-13d9-4263-86f9-6ed6b5adf9b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.667328 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.667038    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-certs" pod "50305fbb-13d9-4263-86f9-6ed6b5adf9b3" (UID: "50305fbb-13d9-4263-86f9-6ed6b5adf9b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.667328 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.667171    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-credential-socket" pod "50305fbb-13d9-4263-86f9-6ed6b5adf9b3" (UID: "50305fbb-13d9-4263-86f9-6ed6b5adf9b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.667487 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.667370    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istiod-ca-cert" pod "50305fbb-13d9-4263-86f9-6ed6b5adf9b3" (UID: "50305fbb-13d9-4263-86f9-6ed6b5adf9b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:00:15.667759 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.667674    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-socket" pod "50305fbb-13d9-4263-86f9-6ed6b5adf9b3" (UID: "50305fbb-13d9-4263-86f9-6ed6b5adf9b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.669222 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.669194    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-token" pod "50305fbb-13d9-4263-86f9-6ed6b5adf9b3" (UID: "50305fbb-13d9-4263-86f9-6ed6b5adf9b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:00:15.669728 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.669704    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-kube-api-access-m48vq" pod "50305fbb-13d9-4263-86f9-6ed6b5adf9b3" (UID: "50305fbb-13d9-4263-86f9-6ed6b5adf9b3"). InnerVolumeSpecName "kube-api-access-m48vq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:00:15.670057 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.670028    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-podinfo" pod "50305fbb-13d9-4263-86f9-6ed6b5adf9b3" (UID: "50305fbb-13d9-4263-86f9-6ed6b5adf9b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 16:00:15.671335 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.671306    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-envoy" pod "50305fbb-13d9-4263-86f9-6ed6b5adf9b3" (UID: "50305fbb-13d9-4263-86f9-6ed6b5adf9b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:00:15.766622 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.766530    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m48vq\" (UniqueName: \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-kube-api-access-m48vq\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.766622 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.766576    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.766622 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.766596    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.766622 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.766613    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.766847 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.766627    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.766847 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.766640    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.766847 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.766654    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.766847 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.766667    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:15.766847 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:15.766680    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50305fbb-13d9-4263-86f9-6ed6b5adf9b3-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:16.371887 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.371858    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"
Oct 30 16:00:16.372537 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.371859    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8" event={"ID":"50305fbb-13d9-4263-86f9-6ed6b5adf9b3","Type":"ContainerDied","Data":"7750dba4e7c8eda981da36419a3e55e349dfd179150b6f6b9cf91241ed3ed126"}
Oct 30 16:00:16.372693 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.372581    4585 scope.go:122] "RemoveContainer" containerID="adcdafd246cf0ec79c776b1097f171f65a8a9efb60beb5eb68fc8b725cbf7a96"
Oct 30 16:00:16.374258 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.374227    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" event={"ID":"876862ba-27c9-4c8b-b440-93ec2241b948","Type":"ContainerDied","Data":"878e9eeb8399238701fd3e8e62f708c1d13edfe91579bf0bbd37cb8cda2efef0"}
Oct 30 16:00:16.374513 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.374493    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"
Oct 30 16:00:16.381252 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.381226    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl" event={"ID":"809eb8ab-fb39-471e-a770-3eaea55689a9","Type":"ContainerDied","Data":"a059045dcf701083e11ce48558eb60086301b654e9a9d2bf0d16d8684e1c10ad"}
Oct 30 16:00:16.381368 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.381318    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"
Oct 30 16:00:16.393332 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.393306    4585 scope.go:122] "RemoveContainer" containerID="cf199c732f2ba03c1cbc0781f88a36353c7972bb86030c70603749dcabaf97dd"
Oct 30 16:00:16.404085 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.404060    4585 scope.go:122] "RemoveContainer" containerID="d9548d16c12bcfc4480db3c701fc64569c8926c4803e2384a4f7409ac110cbc5"
Oct 30 16:00:16.416720 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.416689    4585 patch_prober.go:29] interesting pod/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 16:00:16.416852 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.416762    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 16:00:16.452445 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.448139    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"]
Oct 30 16:00:16.452445 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.452179    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-gtqw8"]
Oct 30 16:00:16.501384 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.499442    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"]
Oct 30 16:00:16.501384 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.500568    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-2c4vl"]
Oct 30 16:00:16.536385 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.534130    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"]
Oct 30 16:00:16.550397 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.546334    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-ba9db9-openshift-default-8b9b5cf77-fz5rw"]
Oct 30 16:00:16.699372 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.696844    4585 scope.go:122] "RemoveContainer" containerID="0b9d846721b78c547a924dcc9a2b1d31d312c317f8f612b08020f60675a60c6d"
Oct 30 16:00:16.707375 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.706806    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" path="/var/lib/kubelet/pods/50305fbb-13d9-4263-86f9-6ed6b5adf9b3/volumes"
Oct 30 16:00:16.707539 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.707493    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" path="/var/lib/kubelet/pods/809eb8ab-fb39-471e-a770-3eaea55689a9/volumes"
Oct 30 16:00:16.708134 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:16.708109    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" path="/var/lib/kubelet/pods/876862ba-27c9-4c8b-b440-93ec2241b948/volumes"
Oct 30 16:00:17.426198 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:17.426147    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" event={"ID":"366f8b79-c76d-4d7c-a94f-95f5e4e4be26","Type":"ContainerStarted","Data":"dacf2f5147f9a6f4bbc0836ac861c4947c0aef994e358b3e7afca854750b0c44"}
Oct 30 16:00:18.152557 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:00:18.151342    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:00:18.441630 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:18.441526    4585 generic.go:356] "Generic (PLEG): container finished" podID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerID="dacf2f5147f9a6f4bbc0836ac861c4947c0aef994e358b3e7afca854750b0c44" exitCode=60
Oct 30 16:00:18.441630 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:18.441560    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" event={"ID":"366f8b79-c76d-4d7c-a94f-95f5e4e4be26","Type":"ContainerDied","Data":"dacf2f5147f9a6f4bbc0836ac861c4947c0aef994e358b3e7afca854750b0c44"}
Oct 30 16:00:18.442170 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:18.441660    4585 scope.go:122] "RemoveContainer" containerID="0b9d846721b78c547a924dcc9a2b1d31d312c317f8f612b08020f60675a60c6d"
Oct 30 16:00:18.442170 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:18.442089    4585 scope.go:122] "RemoveContainer" containerID="dacf2f5147f9a6f4bbc0836ac861c4947c0aef994e358b3e7afca854750b0c44"
Oct 30 16:00:18.442460 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:00:18.442428    4585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889600-np7nn_odh-ai-gateway-infra(366f8b79-c76d-4d7c-a94f-95f5e4e4be26)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26"
Oct 30 16:00:18.578768 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:18.578732    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"]
Oct 30 16:00:19.596248 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:19.596223    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"
Oct 30 16:00:19.665464 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:19.665427    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-maas-api-service-ca\") pod \"366f8b79-c76d-4d7c-a94f-95f5e4e4be26\" (UID: \"366f8b79-c76d-4d7c-a94f-95f5e4e4be26\") "
Oct 30 16:00:19.665658 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:19.665636    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-kube-api-access-zr9g8\" (UniqueName: \"kubernetes.io/projected/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-kube-api-access-zr9g8\") pod \"366f8b79-c76d-4d7c-a94f-95f5e4e4be26\" (UID: \"366f8b79-c76d-4d7c-a94f-95f5e4e4be26\") "
Oct 30 16:00:19.665772 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:19.665751    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-maas-api-service-ca" pod "366f8b79-c76d-4d7c-a94f-95f5e4e4be26" (UID: "366f8b79-c76d-4d7c-a94f-95f5e4e4be26"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:00:19.665971 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:19.665957    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-maas-api-service-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:19.667758 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:19.667736    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-kube-api-access-zr9g8" pod "366f8b79-c76d-4d7c-a94f-95f5e4e4be26" (UID: "366f8b79-c76d-4d7c-a94f-95f5e4e4be26"). InnerVolumeSpecName "kube-api-access-zr9g8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:00:19.766823 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:19.766727    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr9g8\" (UniqueName: \"kubernetes.io/projected/366f8b79-c76d-4d7c-a94f-95f5e4e4be26-kube-api-access-zr9g8\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:00:20.455856 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:20.455821    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn" event={"ID":"366f8b79-c76d-4d7c-a94f-95f5e4e4be26","Type":"ContainerDied","Data":"7047dde17046622d33824f47ce640133f4ade6b0999aa764a47a2c9e9f3fa501"}
Oct 30 16:00:20.455856 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:20.455852    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"
Oct 30 16:00:20.456105 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:20.455889    4585 scope.go:122] "RemoveContainer" containerID="dacf2f5147f9a6f4bbc0836ac861c4947c0aef994e358b3e7afca854750b0c44"
Oct 30 16:00:20.519492 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:20.519460    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"]
Oct 30 16:00:20.531402 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:20.531375    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889600-np7nn"]
Oct 30 16:00:20.700048 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:00:20.700018    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" path="/var/lib/kubelet/pods/366f8b79-c76d-4d7c-a94f-95f5e4e4be26/volumes"
Oct 30 16:00:47.961112 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:00:47.961076    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:01:17.961862 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:01:17.961815    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:01:30.050931 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:01:30.050897    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"]
Oct 30 16:01:30.062965 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:01:30.062939    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cb15f4b0-p5th6"]
Oct 30 16:01:30.701051 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:01:30.701019    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86e0f877-9d9f-4716-8e8a-707d2096b466" path="/var/lib/kubelet/pods/86e0f877-9d9f-4716-8e8a-707d2096b466/volumes"
Oct 30 16:01:48.010135 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:01:48.010089    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:02:17.963736 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:02:17.963691    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:02:47.959915 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:02:47.959872    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:03:11.533528 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:11.533497    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 16:03:11.538619 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:11.538597    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 16:03:11.889503 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:11.889424    4585 scope.go:122] "RemoveContainer" containerID="c033b8c410842cf42a30294c4c996ec9674cb0a65e22b2d36963d8e3bee57a80"
Oct 30 16:03:18.137100 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:03:18.137055    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:03:18.725369 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:18.724485    4585 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 16:03:18.725369 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:18.724571    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-9dccdc9b7-rtvxh" podUID="619e7d6d-2940-4118-81a9-13a16430f65b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 16:03:22.055239 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:22.055193    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"]
Oct 30 16:03:22.066978 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:22.066940    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc62ed0f-fzxcz"]
Oct 30 16:03:22.700751 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:22.700717    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96fb8d85-a90d-4c9f-9772-5da645f27721" path="/var/lib/kubelet/pods/96fb8d85-a90d-4c9f-9772-5da645f27721/volumes"
Oct 30 16:03:34.028824 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:34.028791    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"]
Oct 30 16:03:34.040688 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:34.040663    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7597c6b3-qh9fp"]
Oct 30 16:03:34.700941 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:03:34.700908    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc082cf8-74f3-4067-a26e-ceb24c6c88da" path="/var/lib/kubelet/pods/cc082cf8-74f3-4067-a26e-ceb24c6c88da/volumes"
Oct 30 16:03:47.960494 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:03:47.960456    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:04:01.033602 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:01.033566    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"]
Oct 30 16:04:01.045745 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:01.045713    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"]
Oct 30 16:04:01.047198 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:01.047178    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"]
Oct 30 16:04:01.048415 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:01.048405    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1e54e2f2-6pnfb"]
Oct 30 16:04:01.049607 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:01.049588    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a5343dac-87pgc"]
Oct 30 16:04:01.057454 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:01.057430    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-71729861-4mzwc"]
Oct 30 16:04:02.700165 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:02.700127    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41f752fd-9135-461f-a763-a5fdb474161f" path="/var/lib/kubelet/pods/41f752fd-9135-461f-a763-a5fdb474161f/volumes"
Oct 30 16:04:02.700569 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:02.700559    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2" path="/var/lib/kubelet/pods/5dcdb205-4db0-4ef8-a2d8-b8b9e5597ed2/volumes"
Oct 30 16:04:02.700928 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:02.700914    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5ca140b-084b-4889-9881-f28f7dd09b56" path="/var/lib/kubelet/pods/d5ca140b-084b-4889-9881-f28f7dd09b56/volumes"
Oct 30 16:04:09.029862 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:09.029820    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"]
Oct 30 16:04:09.042172 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:09.042140    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e786081d-rjxfx"]
Oct 30 16:04:10.701134 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:10.701102    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc457368-452b-4b4b-aab5-9250e11f056e" path="/var/lib/kubelet/pods/bc457368-452b-4b4b-aab5-9250e11f056e/volumes"
Oct 30 16:04:17.961609 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:04:17.961571    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:04:47.960809 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:04:47.960731    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:04:48.112185 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.112149    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"]
Oct 30 16:04:48.112412 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.112378    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f" podUID="311f2664-3b46-4ff7-bec0-b553d0204153" containerName="discovery" containerID="cri-o://45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab" gracePeriod=30
Oct 30 16:04:48.372155 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.372130    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 16:04:48.501100 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.501006    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-kube-api-access-pnrts\" (UniqueName: \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-kube-api-access-pnrts\") pod \"311f2664-3b46-4ff7-bec0-b553d0204153\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") "
Oct 30 16:04:48.501100 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.501067    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-dns-cert\") pod \"311f2664-3b46-4ff7-bec0-b553d0204153\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") "
Oct 30 16:04:48.501100 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.501098    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-cacerts\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-cacerts\") pod \"311f2664-3b46-4ff7-bec0-b553d0204153\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") "
Oct 30 16:04:48.501433 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.501167    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/311f2664-3b46-4ff7-bec0-b553d0204153-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/311f2664-3b46-4ff7-bec0-b553d0204153-local-certs\") pod \"311f2664-3b46-4ff7-bec0-b553d0204153\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") "
Oct 30 16:04:48.501433 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.501192    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-istio-token\" (UniqueName: \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-istio-token\") pod \"311f2664-3b46-4ff7-bec0-b553d0204153\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") "
Oct 30 16:04:48.501433 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.501214    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-kubeconfig\") pod \"311f2664-3b46-4ff7-bec0-b553d0204153\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") "
Oct 30 16:04:48.501433 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.501291    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-ca-configmap\") pod \"311f2664-3b46-4ff7-bec0-b553d0204153\" (UID: \"311f2664-3b46-4ff7-bec0-b553d0204153\") "
Oct 30 16:04:48.502029 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.501998    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-ca-configmap" pod "311f2664-3b46-4ff7-bec0-b553d0204153" (UID: "311f2664-3b46-4ff7-bec0-b553d0204153"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:04:48.503597 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.503566    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-dns-cert" pod "311f2664-3b46-4ff7-bec0-b553d0204153" (UID: "311f2664-3b46-4ff7-bec0-b553d0204153"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:04:48.503712 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.503563    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-istio-token" pod "311f2664-3b46-4ff7-bec0-b553d0204153" (UID: "311f2664-3b46-4ff7-bec0-b553d0204153"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:04:48.503865 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.503843    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311f2664-3b46-4ff7-bec0-b553d0204153-local-certs" pod "311f2664-3b46-4ff7-bec0-b553d0204153" (UID: "311f2664-3b46-4ff7-bec0-b553d0204153"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:04:48.503960 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.503842    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-kube-api-access-pnrts" pod "311f2664-3b46-4ff7-bec0-b553d0204153" (UID: "311f2664-3b46-4ff7-bec0-b553d0204153"). InnerVolumeSpecName "kube-api-access-pnrts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:04:48.503960 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.503889    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-cacerts" pod "311f2664-3b46-4ff7-bec0-b553d0204153" (UID: "311f2664-3b46-4ff7-bec0-b553d0204153"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:04:48.504065 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.504043    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-kubeconfig" pod "311f2664-3b46-4ff7-bec0-b553d0204153" (UID: "311f2664-3b46-4ff7-bec0-b553d0204153"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:04:48.602471 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.602429    4585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-ca-configmap\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:04:48.602471 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.602460    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnrts\" (UniqueName: \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-kube-api-access-pnrts\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:04:48.602471 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.602469    4585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-csr-dns-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:04:48.602471 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.602478    4585 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-cacerts\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:04:48.602471 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.602487    4585 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/311f2664-3b46-4ff7-bec0-b553d0204153-local-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:04:48.602868 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.602495    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/311f2664-3b46-4ff7-bec0-b553d0204153-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:04:48.602868 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.602502    4585 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/311f2664-3b46-4ff7-bec0-b553d0204153-istio-kubeconfig\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:04:48.869920 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.869826    4585 generic.go:356] "Generic (PLEG): container finished" podID="311f2664-3b46-4ff7-bec0-b553d0204153" containerID="45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab" exitCode=0
Oct 30 16:04:48.869920 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.869879    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"
Oct 30 16:04:48.869920 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.869903    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f" event={"ID":"311f2664-3b46-4ff7-bec0-b553d0204153","Type":"ContainerDied","Data":"45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab"}
Oct 30 16:04:48.870186 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.869965    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f" event={"ID":"311f2664-3b46-4ff7-bec0-b553d0204153","Type":"ContainerDied","Data":"646aa2924b60f0d5d705916981d1af45a3a188848b8b251924d84794e099e030"}
Oct 30 16:04:48.870186 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.869986    4585 scope.go:122] "RemoveContainer" containerID="45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab"
Oct 30 16:04:48.879430 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.879411    4585 scope.go:122] "RemoveContainer" containerID="45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab"
Oct 30 16:04:48.879771 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:04:48.879749    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab\": container with ID starting with 45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab not found: ID does not exist" containerID="45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab"
Oct 30 16:04:48.879883 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.879778    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab"} err="failed to get container status \"45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab\": rpc error: code = NotFound desc = could not find container \"45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab\": container with ID starting with 45985be7c816f72a55b7ef0fb73bfa8cf5e33dc00095cccd87f4ea126cd1b9ab not found: ID does not exist"
Oct 30 16:04:48.938276 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.938242    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"]
Oct 30 16:04:48.980071 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:48.980038    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4ks2f"]
Oct 30 16:04:50.700811 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:50.700778    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="311f2664-3b46-4ff7-bec0-b553d0204153" path="/var/lib/kubelet/pods/311f2664-3b46-4ff7-bec0-b553d0204153/volumes"
Oct 30 16:04:51.040606 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:51.040515    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"]
Oct 30 16:04:51.051974 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:51.051946    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc71e045-882pl"]
Oct 30 16:04:52.699878 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:52.699843    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee" path="/var/lib/kubelet/pods/1f21b8b8-8a1a-4bdc-bfb7-4c96c09cb0ee/volumes"
Oct 30 16:04:57.875683 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.875644    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"]
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.875985    4585 cpu_manager.go:414] "containerMap: removing container" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.875996    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876007    4585 cpu_manager.go:414] "containerMap: removing container" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerName="cleanup"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876013    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerName="cleanup"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876031    4585 cpu_manager.go:414] "containerMap: removing container" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876037    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876044    4585 cpu_manager.go:414] "containerMap: removing container" podUID="311f2664-3b46-4ff7-bec0-b553d0204153" containerName="discovery"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876049    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="311f2664-3b46-4ff7-bec0-b553d0204153" containerName="discovery"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876059    4585 cpu_manager.go:414] "containerMap: removing container" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerName="cleanup"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876064    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerName="cleanup"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876078    4585 cpu_manager.go:414] "containerMap: removing container" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876083    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876127    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerName="cleanup"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876136    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="50305fbb-13d9-4263-86f9-6ed6b5adf9b3" containerName="istio-proxy"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876143    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="311f2664-3b46-4ff7-bec0-b553d0204153" containerName="discovery"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876149    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerName="cleanup"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876155    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876161    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="809eb8ab-fb39-471e-a770-3eaea55689a9" containerName="istio-proxy"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876166    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerName="cleanup"
Oct 30 16:04:57.876169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.876172    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy"
Oct 30 16:04:57.881103 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.881084    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:57.899136 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.899108    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"]
Oct 30 16:04:57.908729 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.908703    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 16:04:57.908888 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.908734    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 16:04:57.908888 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.908753    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 16:04:57.908888 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.908806    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 16:04:57.997446 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.997410    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/169cd917-64fb-4560-85fd-b238ae572251-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-v4knb\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:57.997612 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.997528    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/169cd917-64fb-4560-85fd-b238ae572251-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-v4knb\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:57.997612 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:57.997563    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4j7v\" (UniqueName: \"kubernetes.io/projected/169cd917-64fb-4560-85fd-b238ae572251-kube-api-access-w4j7v\") pod \"maas-api-revoke-sub-3d4156f1-v4knb\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:58.098869 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.098828    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/169cd917-64fb-4560-85fd-b238ae572251-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-v4knb\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:58.098869 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.098871    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4j7v\" (UniqueName: \"kubernetes.io/projected/169cd917-64fb-4560-85fd-b238ae572251-kube-api-access-w4j7v\") pod \"maas-api-revoke-sub-3d4156f1-v4knb\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:58.099092 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.098945    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/169cd917-64fb-4560-85fd-b238ae572251-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-v4knb\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:58.099631 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.099608    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/169cd917-64fb-4560-85fd-b238ae572251-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-v4knb\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:58.101177 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.101154    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/169cd917-64fb-4560-85fd-b238ae572251-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-v4knb\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:58.122189 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.122161    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4j7v\" (UniqueName: \"kubernetes.io/projected/169cd917-64fb-4560-85fd-b238ae572251-kube-api-access-w4j7v\") pod \"maas-api-revoke-sub-3d4156f1-v4knb\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:58.193995 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.193962    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:04:58.492889 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.492864    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"]
Oct 30 16:04:58.494333 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:04:58.494309    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod169cd917_64fb_4560_85fd_b238ae572251.slice/crio-feed8262e3f739fba2f45a2b2cac0da65fe0b78599da26cb296dae1cb856ef6e WatchSource:0}: Error finding container feed8262e3f739fba2f45a2b2cac0da65fe0b78599da26cb296dae1cb856ef6e: Status 404 returned error can't find the container with id feed8262e3f739fba2f45a2b2cac0da65fe0b78599da26cb296dae1cb856ef6e
Oct 30 16:04:58.919328 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.919292    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb" event={"ID":"169cd917-64fb-4560-85fd-b238ae572251","Type":"ContainerStarted","Data":"e1ffce583acd3a0e0f77268985dc08c97906c179e665534f26c502927c780ef3"}
Oct 30 16:04:58.919830 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.919341    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb" event={"ID":"169cd917-64fb-4560-85fd-b238ae572251","Type":"ContainerStarted","Data":"feed8262e3f739fba2f45a2b2cac0da65fe0b78599da26cb296dae1cb856ef6e"}
Oct 30 16:04:58.960735 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:58.960681    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb" podStartSLOduration=1.960666671 podStartE2EDuration="1.960666671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:04:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:04:58.959180839 +0000 UTC m=+1608.394785187" watchObservedRunningTime="2026-10-30 16:04:58.960666671 +0000 UTC m=+1608.396271019"
Oct 30 16:04:59.925586 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:59.925554    4585 generic.go:356] "Generic (PLEG): container finished" podID="169cd917-64fb-4560-85fd-b238ae572251" containerID="e1ffce583acd3a0e0f77268985dc08c97906c179e665534f26c502927c780ef3" exitCode=0
Oct 30 16:04:59.926054 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:04:59.925635    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb" event={"ID":"169cd917-64fb-4560-85fd-b238ae572251","Type":"ContainerDied","Data":"e1ffce583acd3a0e0f77268985dc08c97906c179e665534f26c502927c780ef3"}
Oct 30 16:05:01.062893 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.062867    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:05:01.141902 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.141859    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/169cd917-64fb-4560-85fd-b238ae572251-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/169cd917-64fb-4560-85fd-b238ae572251-maas-api-xks-ca\") pod \"169cd917-64fb-4560-85fd-b238ae572251\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") "
Oct 30 16:05:01.142078 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.142017    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/169cd917-64fb-4560-85fd-b238ae572251-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/169cd917-64fb-4560-85fd-b238ae572251-maas-api-openshift-ca\") pod \"169cd917-64fb-4560-85fd-b238ae572251\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") "
Oct 30 16:05:01.142121 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.142073    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/169cd917-64fb-4560-85fd-b238ae572251-kube-api-access-w4j7v\" (UniqueName: \"kubernetes.io/projected/169cd917-64fb-4560-85fd-b238ae572251-kube-api-access-w4j7v\") pod \"169cd917-64fb-4560-85fd-b238ae572251\" (UID: \"169cd917-64fb-4560-85fd-b238ae572251\") "
Oct 30 16:05:01.142391 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.142362    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/169cd917-64fb-4560-85fd-b238ae572251-maas-api-openshift-ca" pod "169cd917-64fb-4560-85fd-b238ae572251" (UID: "169cd917-64fb-4560-85fd-b238ae572251"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:05:01.143914 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.143889    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/169cd917-64fb-4560-85fd-b238ae572251-maas-api-xks-ca" pod "169cd917-64fb-4560-85fd-b238ae572251" (UID: "169cd917-64fb-4560-85fd-b238ae572251"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:05:01.144243 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.144216    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/169cd917-64fb-4560-85fd-b238ae572251-kube-api-access-w4j7v" pod "169cd917-64fb-4560-85fd-b238ae572251" (UID: "169cd917-64fb-4560-85fd-b238ae572251"). InnerVolumeSpecName "kube-api-access-w4j7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:05:01.243139 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.243049    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/169cd917-64fb-4560-85fd-b238ae572251-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:05:01.243139 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.243080    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/169cd917-64fb-4560-85fd-b238ae572251-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:05:01.243139 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.243091    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4j7v\" (UniqueName: \"kubernetes.io/projected/169cd917-64fb-4560-85fd-b238ae572251-kube-api-access-w4j7v\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:05:01.935979 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.935949    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"
Oct 30 16:05:01.936167 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.935990    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb" event={"ID":"169cd917-64fb-4560-85fd-b238ae572251","Type":"ContainerDied","Data":"feed8262e3f739fba2f45a2b2cac0da65fe0b78599da26cb296dae1cb856ef6e"}
Oct 30 16:05:01.936167 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:01.936040    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="feed8262e3f739fba2f45a2b2cac0da65fe0b78599da26cb296dae1cb856ef6e"
Oct 30 16:05:11.921949 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:11.921910    4585 scope.go:122] "RemoveContainer" containerID="2c593d27b6c4d70e35c557189de58d6661aa58d7a8fc965fdf8eb80394a1f03f"
Oct 30 16:05:11.930111 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:11.930090    4585 scope.go:122] "RemoveContainer" containerID="65ea9172d926eab59e88c9af2d596d6ac994d992d9c21935e1778df523ceef91"
Oct 30 16:05:11.938913 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:11.938890    4585 scope.go:122] "RemoveContainer" containerID="fb82fd4f5e5a3fad62b9968dfdbefa2e67671ca994866c939c9bd09586e42a63"
Oct 30 16:05:11.947464 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:11.947445    4585 scope.go:122] "RemoveContainer" containerID="c3c3cdf2361ceade769aa5026205a4116a9fd6344eab4907dbc3de01786dc095"
Oct 30 16:05:11.954771 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:11.954752    4585 scope.go:122] "RemoveContainer" containerID="bc95983449ddd9fce56c43f181cbeb885a953d952df5c8feee271047b3577ef5"
Oct 30 16:05:11.961823 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:11.961804    4585 scope.go:122] "RemoveContainer" containerID="1788b6d30b4e5094828ce9a4f506c060d586572c76e0c5a6fdb70b07c48d4c78"
Oct 30 16:05:11.969140 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:11.969106    4585 scope.go:122] "RemoveContainer" containerID="2eedf1beaf6b440398b6d324ba64d36045ea7fdf84017be0948c920cb89b2539"
Oct 30 16:05:17.963755 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:05:17.963714    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:05:19.885363 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.885317    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"]
Oct 30 16:05:19.885755 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.885700    4585 cpu_manager.go:414] "containerMap: removing container" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy"
Oct 30 16:05:19.885755 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.885713    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="876862ba-27c9-4c8b-b440-93ec2241b948" containerName="istio-proxy"
Oct 30 16:05:19.885755 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.885725    4585 cpu_manager.go:414] "containerMap: removing container" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerName="cleanup"
Oct 30 16:05:19.885755 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.885731    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="366f8b79-c76d-4d7c-a94f-95f5e4e4be26" containerName="cleanup"
Oct 30 16:05:19.885755 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.885743    4585 cpu_manager.go:414] "containerMap: removing container" podUID="169cd917-64fb-4560-85fd-b238ae572251" containerName="revoke-keys"
Oct 30 16:05:19.885755 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.885749    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="169cd917-64fb-4560-85fd-b238ae572251" containerName="revoke-keys"
Oct 30 16:05:19.885938 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.885810    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="169cd917-64fb-4560-85fd-b238ae572251" containerName="revoke-keys"
Oct 30 16:05:19.890647 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.890621    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:19.913335 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.913311    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-6a2c74-openshift-default-dockercfg-9rrcx\""
Oct 30 16:05:19.990817 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:19.990772    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"]
Oct 30 16:05:20.023731 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.023696    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-data\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.023731 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.023736    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scflk\" (UniqueName: \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-kube-api-access-scflk\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.024006 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.023760    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-credential-socket\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.024006 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.023886    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-envoy\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.024006 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.023911    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-socket\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.024198 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.024016    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-token\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.024198 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.024035    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68179532-41f3-49f8-b3b1-d8391d73ac2e-istiod-ca-cert\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.024198 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.024060    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-podinfo\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.024198 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.024081    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-certs\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125239 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125200    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-podinfo\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125239 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125241    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-certs\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125514 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125269    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-data\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125514 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125294    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scflk\" (UniqueName: \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-kube-api-access-scflk\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125514 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125315    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-credential-socket\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125514 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125378    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-envoy\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125514 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125406    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-socket\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125514 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125497    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-token\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125805 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125523    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68179532-41f3-49f8-b3b1-d8391d73ac2e-istiod-ca-cert\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125805 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125680    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-certs\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125805 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125728    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-data\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.125987 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.125965    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-socket\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.126039 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.126021    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-credential-socket\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.126272 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.126250    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68179532-41f3-49f8-b3b1-d8391d73ac2e-istiod-ca-cert\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.127779 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.127757    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-envoy\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.127927 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.127911    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-podinfo\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.159826 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.159788    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scflk\" (UniqueName: \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-kube-api-access-scflk\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.162106 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.162073    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-token\") pod \"e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") " pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.203830 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.203803    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:20.532169 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.532144    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"]
Oct 30 16:05:20.533890 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:05:20.533850    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68179532_41f3_49f8_b3b1_d8391d73ac2e.slice/crio-a5d41eb12f5e8ffdbf189dd1604097f6e8ebce5707def8790ad65e8842c36b48 WatchSource:0}: Error finding container a5d41eb12f5e8ffdbf189dd1604097f6e8ebce5707def8790ad65e8842c36b48: Status 404 returned error can't find the container with id a5d41eb12f5e8ffdbf189dd1604097f6e8ebce5707def8790ad65e8842c36b48
Oct 30 16:05:20.536044 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.536010    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:05:20.536137 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.536076    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:05:20.536137 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:20.536109    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:05:21.033031 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:21.032999    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" event={"ID":"68179532-41f3-49f8-b3b1-d8391d73ac2e","Type":"ContainerStarted","Data":"05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d"}
Oct 30 16:05:21.033518 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:21.033047    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" event={"ID":"68179532-41f3-49f8-b3b1-d8391d73ac2e","Type":"ContainerStarted","Data":"a5d41eb12f5e8ffdbf189dd1604097f6e8ebce5707def8790ad65e8842c36b48"}
Oct 30 16:05:21.080644 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:21.080589    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" podStartSLOduration=2.080570421 podStartE2EDuration="2.080570421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:05:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:05:21.078615786 +0000 UTC m=+1630.514220136" watchObservedRunningTime="2026-10-30 16:05:21.080570421 +0000 UTC m=+1630.516174769"
Oct 30 16:05:21.204220 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:21.204185    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:21.209455 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:21.209429    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:22.038008 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:22.037974    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:22.038887 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:22.038868    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:24.959557 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:24.959515    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"]
Oct 30 16:05:24.963082 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:24.963059    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:24.986128 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:24.986098    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"]
Oct 30 16:05:24.995811 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:24.995778    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 16:05:24.996053 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:24.995963    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 16:05:24.996053 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:24.996048    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gp5lt\""
Oct 30 16:05:24.996208 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:24.996100    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-6a2c74-serving-cert\""
Oct 30 16:05:24.997249 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:24.997231    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 16:05:25.075807 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.075771    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.075964 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.075827    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtrh2\" (UniqueName: \"kubernetes.io/projected/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-kube-api-access-jtrh2\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.075964 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.075921    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.076059 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.075971    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.177505 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.177470    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.177648 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.177523    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtrh2\" (UniqueName: \"kubernetes.io/projected/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-kube-api-access-jtrh2\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.177648 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.177570    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.177648 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.177631    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.177784 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:05:25.177664    4585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-serving-cert: secret "maas-api-e2e-worker-single-main-6a2c74-serving-cert" not found
Oct 30 16:05:25.177784 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:05:25.177759    4585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-maas-api-tls podName:07a31042-4197-4ccb-9fc6-ac208f9dc4f6 nodeName:}" failed. No retries permitted until 2026-10-30 16:05:25.677736034 +0000 UTC m=+1635.113340380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-maas-api-tls") pod "maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8" (UID: "07a31042-4197-4ccb-9fc6-ac208f9dc4f6") : secret "maas-api-e2e-worker-single-main-6a2c74-serving-cert" not found
Oct 30 16:05:25.178283 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.178258    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.180087 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.180063    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.201034 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.201003    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtrh2\" (UniqueName: \"kubernetes.io/projected/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-kube-api-access-jtrh2\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.681878 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.681839    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.684388 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.684365    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:25.877274 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:25.877232    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:26.175035 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:26.175009    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"]
Oct 30 16:05:26.178171 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:05:26.178139    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07a31042_4197_4ccb_9fc6_ac208f9dc4f6.slice/crio-1cc4f2b3f31847e8a6958708798368bb9934b8a4cbc49cabb61c8072a10e2278 WatchSource:0}: Error finding container 1cc4f2b3f31847e8a6958708798368bb9934b8a4cbc49cabb61c8072a10e2278: Status 404 returned error can't find the container with id 1cc4f2b3f31847e8a6958708798368bb9934b8a4cbc49cabb61c8072a10e2278
Oct 30 16:05:26.179477 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:26.179460    4585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 16:05:27.069005 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:27.068888    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8" event={"ID":"07a31042-4197-4ccb-9fc6-ac208f9dc4f6","Type":"ContainerStarted","Data":"5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4"}
Oct 30 16:05:27.069005 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:27.069000    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8" event={"ID":"07a31042-4197-4ccb-9fc6-ac208f9dc4f6","Type":"ContainerStarted","Data":"1cc4f2b3f31847e8a6958708798368bb9934b8a4cbc49cabb61c8072a10e2278"}
Oct 30 16:05:27.069227 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:27.069032    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:27.127931 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:27.127880    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8" podStartSLOduration=2.636025258 podStartE2EDuration="3.127863057s" totalImagesPullingTime="491.837799ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:05:27.127031711 +0000 UTC m=+1636.562636059" watchObservedRunningTime="2026-10-30 16:05:27.127863057 +0000 UTC m=+1636.563467395"
Oct 30 16:05:33.080017 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:33.079977    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:05:38.080478 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:38.080445    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 16:05:38.080898 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:38.080534    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 16:05:39.126942 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:39.126911    4585 generic.go:356] "Generic (PLEG): container finished" podID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerID="05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d" exitCode=0
Oct 30 16:05:39.127504 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:39.126989    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" event={"ID":"68179532-41f3-49f8-b3b1-d8391d73ac2e","Type":"ContainerDied","Data":"05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d"}
Oct 30 16:05:39.128027 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:39.128005    4585 scope.go:122] "RemoveContainer" containerID="05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d"
Oct 30 16:05:39.129436 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:39.129406    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:05:39.129512 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:39.129477    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:05:39.129551 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:39.129517    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:05:40.155315 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:40.155265    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" event={"ID":"68179532-41f3-49f8-b3b1-d8391d73ac2e","Type":"ContainerStarted","Data":"e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892"}
Oct 30 16:05:40.204230 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:40.204192    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:40.204520 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:40.204500    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 16:05:40.204574 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:40.204544    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 16:05:41.204515 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:41.204479    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 16:05:41.205002 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:41.204538    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 16:05:42.204714 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:42.204684    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 16:05:42.205185 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:42.205159    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 16:05:43.204212 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:43.204174    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 16:05:43.204416 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:43.204238    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 16:05:44.205064 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:44.205026    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 16:05:44.205589 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:44.205091    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 16:05:45.776509 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:45.776475    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:45.777474 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:45.777399    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:45.778164 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:05:45.778146    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:05:47.960192 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:05:47.960158    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:06:12.011299 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:12.011268    4585 scope.go:122] "RemoveContainer" containerID="9a3d33dd16afeb17e96b49b771f3fafabba578405051f8b58e596950776521a1"
Oct 30 16:06:17.960766 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:06:17.960719    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:06:29.521366 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:29.521309    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"]
Oct 30 16:06:29.521847 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:29.521581    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8" podUID="07a31042-4197-4ccb-9fc6-ac208f9dc4f6" containerName="maas-api" containerID="cri-o://5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4" gracePeriod=30
Oct 30 16:06:29.988751 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:29.988722    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:06:30.064963 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.064873    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-kube-api-access-jtrh2\" (UniqueName: \"kubernetes.io/projected/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-kube-api-access-jtrh2\") pod \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") "
Oct 30 16:06:30.064963 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.064947    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-metrics-certs\") pod \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") "
Oct 30 16:06:30.065212 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.064972    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-maas-api-tls\") pod \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") "
Oct 30 16:06:30.065212 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.065036    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-openshift-service-ca\") pod \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\" (UID: \"07a31042-4197-4ccb-9fc6-ac208f9dc4f6\") "
Oct 30 16:06:30.065536 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.065506    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-openshift-service-ca" pod "07a31042-4197-4ccb-9fc6-ac208f9dc4f6" (UID: "07a31042-4197-4ccb-9fc6-ac208f9dc4f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:06:30.067157 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.067130    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-maas-api-tls" pod "07a31042-4197-4ccb-9fc6-ac208f9dc4f6" (UID: "07a31042-4197-4ccb-9fc6-ac208f9dc4f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:06:30.067257 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.067171    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-metrics-certs" pod "07a31042-4197-4ccb-9fc6-ac208f9dc4f6" (UID: "07a31042-4197-4ccb-9fc6-ac208f9dc4f6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:06:30.067257 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.067180    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-kube-api-access-jtrh2" pod "07a31042-4197-4ccb-9fc6-ac208f9dc4f6" (UID: "07a31042-4197-4ccb-9fc6-ac208f9dc4f6"). InnerVolumeSpecName "kube-api-access-jtrh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:06:30.166771 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.166735    4585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-metrics-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:30.166771 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.166767    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-maas-api-tls\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:30.166771 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.166776    4585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-openshift-service-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:30.167010 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.166786    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtrh2\" (UniqueName: \"kubernetes.io/projected/07a31042-4197-4ccb-9fc6-ac208f9dc4f6-kube-api-access-jtrh2\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:30.387819 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.387722    4585 generic.go:356] "Generic (PLEG): container finished" podID="07a31042-4197-4ccb-9fc6-ac208f9dc4f6" containerID="5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4" exitCode=0
Oct 30 16:06:30.387819 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.387773    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"
Oct 30 16:06:30.387819 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.387808    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8" event={"ID":"07a31042-4197-4ccb-9fc6-ac208f9dc4f6","Type":"ContainerDied","Data":"5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4"}
Oct 30 16:06:30.388107 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.387866    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8" event={"ID":"07a31042-4197-4ccb-9fc6-ac208f9dc4f6","Type":"ContainerDied","Data":"1cc4f2b3f31847e8a6958708798368bb9934b8a4cbc49cabb61c8072a10e2278"}
Oct 30 16:06:30.388107 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.387894    4585 scope.go:122] "RemoveContainer" containerID="5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4"
Oct 30 16:06:30.399377 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.399339    4585 scope.go:122] "RemoveContainer" containerID="5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4"
Oct 30 16:06:30.399645 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:06:30.399625    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4\": container with ID starting with 5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4 not found: ID does not exist" containerID="5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4"
Oct 30 16:06:30.399702 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.399654    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4"} err="failed to get container status \"5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4\": rpc error: code = NotFound desc = could not find container \"5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4\": container with ID starting with 5789c5d46ac6ddf958fe989e425e626c849eb8d01a6d9f7328200bb8c57abad4 not found: ID does not exist"
Oct 30 16:06:30.446861 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.446828    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"]
Oct 30 16:06:30.458657 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.458622    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-6a2c74-58d7cb7bb4-2g8v8"]
Oct 30 16:06:30.699821 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:30.699788    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07a31042-4197-4ccb-9fc6-ac208f9dc4f6" path="/var/lib/kubelet/pods/07a31042-4197-4ccb-9fc6-ac208f9dc4f6/volumes"
Oct 30 16:06:37.941992 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:37.941946    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"]
Oct 30 16:06:37.942560 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:37.942264    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy" containerID="cri-o://e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892" gracePeriod=30
Oct 30 16:06:39.245552 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.245520    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"]
Oct 30 16:06:39.257994 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.257960    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-v4knb"]
Oct 30 16:06:39.296823 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.296795    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"]
Oct 30 16:06:39.297143 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.297130    4585 cpu_manager.go:414] "containerMap: removing container" podUID="07a31042-4197-4ccb-9fc6-ac208f9dc4f6" containerName="maas-api"
Oct 30 16:06:39.297189 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.297144    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07a31042-4197-4ccb-9fc6-ac208f9dc4f6" containerName="maas-api"
Oct 30 16:06:39.297224 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.297221    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="07a31042-4197-4ccb-9fc6-ac208f9dc4f6" containerName="maas-api"
Oct 30 16:06:39.299247 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.299230    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.327725 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.327695    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"]
Oct 30 16:06:39.329257 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.329232    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 16:06:39.329393 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.329270    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 16:06:39.329393 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.329339    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 16:06:39.329479 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.329463    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 16:06:39.465864 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.465826    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-td9dz\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.466073 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.465892    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-td9dz\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.466073 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.465995    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ck2j\" (UniqueName: \"kubernetes.io/projected/c2bf107f-ff92-4694-9cb2-409bd37eaa89-kube-api-access-8ck2j\") pod \"maas-api-revoke-sub-3d4156f1-td9dz\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.567318 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.567220    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-td9dz\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.567500 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.567381    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-td9dz\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.567500 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.567410    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ck2j\" (UniqueName: \"kubernetes.io/projected/c2bf107f-ff92-4694-9cb2-409bd37eaa89-kube-api-access-8ck2j\") pod \"maas-api-revoke-sub-3d4156f1-td9dz\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.567912 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.567891    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-td9dz\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.569748 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.569723    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-td9dz\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.591399 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.591372    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ck2j\" (UniqueName: \"kubernetes.io/projected/c2bf107f-ff92-4694-9cb2-409bd37eaa89-kube-api-access-8ck2j\") pod \"maas-api-revoke-sub-3d4156f1-td9dz\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.610136 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.610109    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:39.907736 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:39.907710    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"]
Oct 30 16:06:39.908336 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:06:39.908313    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2bf107f_ff92_4694_9cb2_409bd37eaa89.slice/crio-780bddf1f3754e9e9857bc0bfadcd83995b721b71cbaf8d719ec5a499949b004 WatchSource:0}: Error finding container 780bddf1f3754e9e9857bc0bfadcd83995b721b71cbaf8d719ec5a499949b004: Status 404 returned error can't find the container with id 780bddf1f3754e9e9857bc0bfadcd83995b721b71cbaf8d719ec5a499949b004
Oct 30 16:06:40.437772 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:40.437737    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz" event={"ID":"c2bf107f-ff92-4694-9cb2-409bd37eaa89","Type":"ContainerStarted","Data":"4a6978658894596c06e7e1438bb7fc41a0a565675caea0d5d01bbd445a159cf1"}
Oct 30 16:06:40.438185 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:40.437791    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz" event={"ID":"c2bf107f-ff92-4694-9cb2-409bd37eaa89","Type":"ContainerStarted","Data":"780bddf1f3754e9e9857bc0bfadcd83995b721b71cbaf8d719ec5a499949b004"}
Oct 30 16:06:40.479012 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:40.478962    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz" podStartSLOduration=1.4789490619999999 podStartE2EDuration="1.478949062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:06:40.477969043 +0000 UTC m=+1709.913573390" watchObservedRunningTime="2026-10-30 16:06:40.478949062 +0000 UTC m=+1709.914553409"
Oct 30 16:06:40.700424 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:40.700394    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="169cd917-64fb-4560-85fd-b238ae572251" path="/var/lib/kubelet/pods/169cd917-64fb-4560-85fd-b238ae572251/volumes"
Oct 30 16:06:41.443702 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:41.443663    4585 generic.go:356] "Generic (PLEG): container finished" podID="c2bf107f-ff92-4694-9cb2-409bd37eaa89" containerID="4a6978658894596c06e7e1438bb7fc41a0a565675caea0d5d01bbd445a159cf1" exitCode=0
Oct 30 16:06:41.444162 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:41.443708    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz" event={"ID":"c2bf107f-ff92-4694-9cb2-409bd37eaa89","Type":"ContainerDied","Data":"4a6978658894596c06e7e1438bb7fc41a0a565675caea0d5d01bbd445a159cf1"}
Oct 30 16:06:42.576404 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:42.576374    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:42.697256 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:42.697218    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2bf107f-ff92-4694-9cb2-409bd37eaa89-kube-api-access-8ck2j\" (UniqueName: \"kubernetes.io/projected/c2bf107f-ff92-4694-9cb2-409bd37eaa89-kube-api-access-8ck2j\") pod \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") "
Oct 30 16:06:42.697492 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:42.697318    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-openshift-ca\") pod \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") "
Oct 30 16:06:42.697492 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:42.697438    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-xks-ca\") pod \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\" (UID: \"c2bf107f-ff92-4694-9cb2-409bd37eaa89\") "
Oct 30 16:06:42.697709 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:42.697686    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-openshift-ca" pod "c2bf107f-ff92-4694-9cb2-409bd37eaa89" (UID: "c2bf107f-ff92-4694-9cb2-409bd37eaa89"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:06:42.697818 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:42.697800    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:42.699607 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:42.699538    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2bf107f-ff92-4694-9cb2-409bd37eaa89-kube-api-access-8ck2j" pod "c2bf107f-ff92-4694-9cb2-409bd37eaa89" (UID: "c2bf107f-ff92-4694-9cb2-409bd37eaa89"). InnerVolumeSpecName "kube-api-access-8ck2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:06:42.699724 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:42.699621    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-xks-ca" pod "c2bf107f-ff92-4694-9cb2-409bd37eaa89" (UID: "c2bf107f-ff92-4694-9cb2-409bd37eaa89"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:06:42.798521 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:42.798486    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2bf107f-ff92-4694-9cb2-409bd37eaa89-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:42.798521 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:42.798515    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ck2j\" (UniqueName: \"kubernetes.io/projected/c2bf107f-ff92-4694-9cb2-409bd37eaa89-kube-api-access-8ck2j\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:43.180958 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.180933    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:06:43.303534 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303444    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-envoy\") pod \"68179532-41f3-49f8-b3b1-d8391d73ac2e\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") "
Oct 30 16:06:43.303534 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303495    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-credential-socket\") pod \"68179532-41f3-49f8-b3b1-d8391d73ac2e\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") "
Oct 30 16:06:43.303534 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303531    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-kube-api-access-scflk\" (UniqueName: \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-kube-api-access-scflk\") pod \"68179532-41f3-49f8-b3b1-d8391d73ac2e\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") "
Oct 30 16:06:43.303877 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303553    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-data\") pod \"68179532-41f3-49f8-b3b1-d8391d73ac2e\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") "
Oct 30 16:06:43.303877 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303633    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-socket\") pod \"68179532-41f3-49f8-b3b1-d8391d73ac2e\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") "
Oct 30 16:06:43.303877 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303664    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-token\" (UniqueName: \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-token\") pod \"68179532-41f3-49f8-b3b1-d8391d73ac2e\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") "
Oct 30 16:06:43.303877 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303779    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-podinfo\") pod \"68179532-41f3-49f8-b3b1-d8391d73ac2e\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") "
Oct 30 16:06:43.303877 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303819    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68179532-41f3-49f8-b3b1-d8391d73ac2e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68179532-41f3-49f8-b3b1-d8391d73ac2e-istiod-ca-cert\") pod \"68179532-41f3-49f8-b3b1-d8391d73ac2e\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") "
Oct 30 16:06:43.303877 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303862    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-certs\") pod \"68179532-41f3-49f8-b3b1-d8391d73ac2e\" (UID: \"68179532-41f3-49f8-b3b1-d8391d73ac2e\") "
Oct 30 16:06:43.304189 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303859    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-credential-socket" pod "68179532-41f3-49f8-b3b1-d8391d73ac2e" (UID: "68179532-41f3-49f8-b3b1-d8391d73ac2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:06:43.304189 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303947    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-socket" pod "68179532-41f3-49f8-b3b1-d8391d73ac2e" (UID: "68179532-41f3-49f8-b3b1-d8391d73ac2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:06:43.304189 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.303962    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-data" pod "68179532-41f3-49f8-b3b1-d8391d73ac2e" (UID: "68179532-41f3-49f8-b3b1-d8391d73ac2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:06:43.304189 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.304137    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68179532-41f3-49f8-b3b1-d8391d73ac2e-istiod-ca-cert" pod "68179532-41f3-49f8-b3b1-d8391d73ac2e" (UID: "68179532-41f3-49f8-b3b1-d8391d73ac2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:06:43.304189 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.304145    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-certs" pod "68179532-41f3-49f8-b3b1-d8391d73ac2e" (UID: "68179532-41f3-49f8-b3b1-d8391d73ac2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:06:43.304486 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.304304    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68179532-41f3-49f8-b3b1-d8391d73ac2e-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:43.304486 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.304321    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:43.304486 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.304330    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:43.304486 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.304341    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:43.304486 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.304370    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:43.305874 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.305848    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-kube-api-access-scflk" pod "68179532-41f3-49f8-b3b1-d8391d73ac2e" (UID: "68179532-41f3-49f8-b3b1-d8391d73ac2e"). InnerVolumeSpecName "kube-api-access-scflk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:06:43.305992 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.305906    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-podinfo" pod "68179532-41f3-49f8-b3b1-d8391d73ac2e" (UID: "68179532-41f3-49f8-b3b1-d8391d73ac2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 16:06:43.306114 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.306092    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-envoy" pod "68179532-41f3-49f8-b3b1-d8391d73ac2e" (UID: "68179532-41f3-49f8-b3b1-d8391d73ac2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:06:43.306155 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.306118    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-token" pod "68179532-41f3-49f8-b3b1-d8391d73ac2e" (UID: "68179532-41f3-49f8-b3b1-d8391d73ac2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:06:43.405239 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.405209    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:43.405239 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.405234    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:43.405239 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.405244    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scflk\" (UniqueName: \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-kube-api-access-scflk\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:43.405512 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.405253    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68179532-41f3-49f8-b3b1-d8391d73ac2e-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:06:43.453235 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.453199    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"
Oct 30 16:06:43.453235 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.453220    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz" event={"ID":"c2bf107f-ff92-4694-9cb2-409bd37eaa89","Type":"ContainerDied","Data":"780bddf1f3754e9e9857bc0bfadcd83995b721b71cbaf8d719ec5a499949b004"}
Oct 30 16:06:43.453533 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.453272    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="780bddf1f3754e9e9857bc0bfadcd83995b721b71cbaf8d719ec5a499949b004"
Oct 30 16:06:43.454984 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.454954    4585 generic.go:356] "Generic (PLEG): container finished" podID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerID="e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892" exitCode=0
Oct 30 16:06:43.455118 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.454999    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" event={"ID":"68179532-41f3-49f8-b3b1-d8391d73ac2e","Type":"ContainerDied","Data":"e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892"}
Oct 30 16:06:43.455118 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.455010    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"
Oct 30 16:06:43.455118 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.455035    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq" event={"ID":"68179532-41f3-49f8-b3b1-d8391d73ac2e","Type":"ContainerDied","Data":"a5d41eb12f5e8ffdbf189dd1604097f6e8ebce5707def8790ad65e8842c36b48"}
Oct 30 16:06:43.455118 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.455060    4585 scope.go:122] "RemoveContainer" containerID="e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892"
Oct 30 16:06:43.466110 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.466087    4585 scope.go:122] "RemoveContainer" containerID="05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d"
Oct 30 16:06:43.474395 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.474340    4585 scope.go:122] "RemoveContainer" containerID="e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892"
Oct 30 16:06:43.474658 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:06:43.474637    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892\": container with ID starting with e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892 not found: ID does not exist" containerID="e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892"
Oct 30 16:06:43.474739 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.474670    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892"} err="failed to get container status \"e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892\": rpc error: code = NotFound desc = could not find container \"e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892\": container with ID starting with e769466752d86d8d52093d2e4a5251fd4e932fcec2dfc7300906f4a4a1b0c892 not found: ID does not exist"
Oct 30 16:06:43.474739 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.474692    4585 scope.go:122] "RemoveContainer" containerID="05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d"
Oct 30 16:06:43.474945 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:06:43.474927    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d\": container with ID starting with 05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d not found: ID does not exist" containerID="05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d"
Oct 30 16:06:43.474984 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.474952    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d"} err="failed to get container status \"05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d\": rpc error: code = NotFound desc = could not find container \"05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d\": container with ID starting with 05fbd1c19c016c26082308ddc43f3fd0bbac0a0125294d4d85507598bdb42c3d not found: ID does not exist"
Oct 30 16:06:43.524328 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.524293    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"]
Oct 30 16:06:43.530940 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:43.530917    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-6a2c74-openshift-default-5556c584f-p5jgq"]
Oct 30 16:06:44.700961 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:06:44.700931    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" path="/var/lib/kubelet/pods/68179532-41f3-49f8-b3b1-d8391d73ac2e/volumes"
Oct 30 16:06:47.960981 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:06:47.960942    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:07:03.525510 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:03.525471    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"]
Oct 30 16:07:03.525944 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:03.525690    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr" podUID="4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1" containerName="manager" containerID="cri-o://38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1" gracePeriod=10
Oct 30 16:07:03.776557 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:03.776490    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 16:07:03.831559 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:03.831529    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-extensions-socket-volume\") pod \"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1\" (UID: \"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1\") "
Oct 30 16:07:03.831755 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:03.831595    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-kube-api-access-nphf8\" (UniqueName: \"kubernetes.io/projected/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-kube-api-access-nphf8\") pod \"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1\" (UID: \"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1\") "
Oct 30 16:07:03.832010 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:03.831985    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-extensions-socket-volume" pod "4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1" (UID: "4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:07:03.838703 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:03.838665    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-kube-api-access-nphf8" pod "4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1" (UID: "4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1"). InnerVolumeSpecName "kube-api-access-nphf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:07:03.932475 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:03.932442    4585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-extensions-socket-volume\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:07:03.932475 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:03.932473    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nphf8\" (UniqueName: \"kubernetes.io/projected/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1-kube-api-access-nphf8\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:07:04.558394 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:04.558341    4585 generic.go:356] "Generic (PLEG): container finished" podID="4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1" containerID="38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1" exitCode=0
Oct 30 16:07:04.558894 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:04.558401    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"
Oct 30 16:07:04.558894 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:04.558413    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr" event={"ID":"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1","Type":"ContainerDied","Data":"38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1"}
Oct 30 16:07:04.558894 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:04.558467    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr" event={"ID":"4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1","Type":"ContainerDied","Data":"efe2a86d974ca6abd9e7476c53c33377b5b3ab727ef59f0fdec7258a990b0bde"}
Oct 30 16:07:04.558894 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:04.558497    4585 scope.go:122] "RemoveContainer" containerID="38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1"
Oct 30 16:07:04.568927 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:04.568907    4585 scope.go:122] "RemoveContainer" containerID="38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1"
Oct 30 16:07:04.569220 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:07:04.569201    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1\": container with ID starting with 38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1 not found: ID does not exist" containerID="38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1"
Oct 30 16:07:04.569270 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:04.569229    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1"} err="failed to get container status \"38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1\": rpc error: code = NotFound desc = could not find container \"38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1\": container with ID starting with 38eb6794a9604855db94eeb5dd09d77353ea1838d600b388a8caad518f6703a1 not found: ID does not exist"
Oct 30 16:07:04.635155 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:04.630627    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"]
Oct 30 16:07:04.641206 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:04.641173    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-ptltr"]
Oct 30 16:07:04.699491 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:07:04.699457    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1" path="/var/lib/kubelet/pods/4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1/volumes"
Oct 30 16:07:17.960944 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:07:17.960902    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:07:47.961361 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:07:47.961308    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:08:06.681602 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.681567    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"]
Oct 30 16:08:06.682007 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.681911    4585 cpu_manager.go:414] "containerMap: removing container" podUID="c2bf107f-ff92-4694-9cb2-409bd37eaa89" containerName="revoke-keys"
Oct 30 16:08:06.682007 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.681922    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2bf107f-ff92-4694-9cb2-409bd37eaa89" containerName="revoke-keys"
Oct 30 16:08:06.682007 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.681939    4585 cpu_manager.go:414] "containerMap: removing container" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy"
Oct 30 16:08:06.682007 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.681944    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy"
Oct 30 16:08:06.682007 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.681961    4585 cpu_manager.go:414] "containerMap: removing container" podUID="4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1" containerName="manager"
Oct 30 16:08:06.682007 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.681966    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1" containerName="manager"
Oct 30 16:08:06.682200 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.682023    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy"
Oct 30 16:08:06.682200 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.682033    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy"
Oct 30 16:08:06.682200 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.682039    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a4ab6c7-0731-4ea1-84dd-b8aa22e998c1" containerName="manager"
Oct 30 16:08:06.682200 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.682047    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2bf107f-ff92-4694-9cb2-409bd37eaa89" containerName="revoke-keys"
Oct 30 16:08:06.685064 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.685047    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"
Oct 30 16:08:06.715267 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.715237    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-trpmg\""
Oct 30 16:08:06.716656 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.716635    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"]
Oct 30 16:08:06.743064 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.743032    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfsml\" (UniqueName: \"kubernetes.io/projected/1d5cd07c-a347-4580-b819-d7677753c15c-kube-api-access-qfsml\") pod \"kuadrant-operator-controller-manager-57bb9bfff-bsgcr\" (UID: \"1d5cd07c-a347-4580-b819-d7677753c15c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"
Oct 30 16:08:06.743220 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.743097    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d5cd07c-a347-4580-b819-d7677753c15c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-bsgcr\" (UID: \"1d5cd07c-a347-4580-b819-d7677753c15c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"
Oct 30 16:08:06.843702 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.843664    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d5cd07c-a347-4580-b819-d7677753c15c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-bsgcr\" (UID: \"1d5cd07c-a347-4580-b819-d7677753c15c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"
Oct 30 16:08:06.843886 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.843768    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfsml\" (UniqueName: \"kubernetes.io/projected/1d5cd07c-a347-4580-b819-d7677753c15c-kube-api-access-qfsml\") pod \"kuadrant-operator-controller-manager-57bb9bfff-bsgcr\" (UID: \"1d5cd07c-a347-4580-b819-d7677753c15c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"
Oct 30 16:08:06.844067 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.844045    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d5cd07c-a347-4580-b819-d7677753c15c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-bsgcr\" (UID: \"1d5cd07c-a347-4580-b819-d7677753c15c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"
Oct 30 16:08:06.868987 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.868949    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfsml\" (UniqueName: \"kubernetes.io/projected/1d5cd07c-a347-4580-b819-d7677753c15c-kube-api-access-qfsml\") pod \"kuadrant-operator-controller-manager-57bb9bfff-bsgcr\" (UID: \"1d5cd07c-a347-4580-b819-d7677753c15c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"
Oct 30 16:08:06.997723 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:06.997633    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"
Oct 30 16:08:07.326828 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:07.326741    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"]
Oct 30 16:08:07.339210 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:08:07.339178    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d5cd07c_a347_4580_b819_d7677753c15c.slice/crio-6113e7b959d028a08a943597a5258bab464613e1cde649eeb43e0ef6aad2cafa WatchSource:0}: Error finding container 6113e7b959d028a08a943597a5258bab464613e1cde649eeb43e0ef6aad2cafa: Status 404 returned error can't find the container with id 6113e7b959d028a08a943597a5258bab464613e1cde649eeb43e0ef6aad2cafa
Oct 30 16:08:07.834500 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:07.834464    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr" event={"ID":"1d5cd07c-a347-4580-b819-d7677753c15c","Type":"ContainerStarted","Data":"86ebed54db6a5aa041a3b7e4923d373011d49447232526fe6b1e782410a87a03"}
Oct 30 16:08:07.834929 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:07.834524    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr" event={"ID":"1d5cd07c-a347-4580-b819-d7677753c15c","Type":"ContainerStarted","Data":"6113e7b959d028a08a943597a5258bab464613e1cde649eeb43e0ef6aad2cafa"}
Oct 30 16:08:07.834929 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:07.834555    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"
Oct 30 16:08:07.972407 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:07.972342    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr" podStartSLOduration=1.97233014 podStartE2EDuration="1.97233014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:08:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:08:07.949205309 +0000 UTC m=+1797.384809657" watchObservedRunningTime="2026-10-30 16:08:07.97233014 +0000 UTC m=+1797.407934488"
Oct 30 16:08:11.560224 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:11.560191    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 16:08:11.565748 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:11.565725    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 16:08:17.962090 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:08:17.962054    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:08:18.841331 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:18.841295    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-bsgcr"
Oct 30 16:08:45.654882 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.654844    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"]
Oct 30 16:08:45.655266 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.655198    4585 cpu_manager.go:414] "containerMap: removing container" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy"
Oct 30 16:08:45.655266 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.655208    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68179532-41f3-49f8-b3b1-d8391d73ac2e" containerName="istio-proxy"
Oct 30 16:08:45.659431 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.657782    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:45.677638 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.677609    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"]
Oct 30 16:08:45.687769 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.687743    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 16:08:45.687769 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.687743    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 16:08:45.687769 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.687742    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 16:08:45.687977 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.687804    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 16:08:45.708018 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.707987    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sfhl\" (UniqueName: \"kubernetes.io/projected/61ca2cb3-f6e9-402e-aba6-48a14871d95c-kube-api-access-7sfhl\") pod \"maas-api-revoke-sub-d2bc93bc-dnlk2\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:45.708174 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.708054    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-dnlk2\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:45.708286 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.708201    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-dnlk2\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:45.808930 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.808887    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-dnlk2\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:45.808930 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.808938    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-dnlk2\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:45.809204 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.808990    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sfhl\" (UniqueName: \"kubernetes.io/projected/61ca2cb3-f6e9-402e-aba6-48a14871d95c-kube-api-access-7sfhl\") pod \"maas-api-revoke-sub-d2bc93bc-dnlk2\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:45.809718 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.809693    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-dnlk2\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:45.811323 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.811300    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-dnlk2\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:45.832701 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.832671    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sfhl\" (UniqueName: \"kubernetes.io/projected/61ca2cb3-f6e9-402e-aba6-48a14871d95c-kube-api-access-7sfhl\") pod \"maas-api-revoke-sub-d2bc93bc-dnlk2\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:45.974151 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:45.974070    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:46.269830 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:46.269798    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"]
Oct 30 16:08:46.271253 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:08:46.271226    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ca2cb3_f6e9_402e_aba6_48a14871d95c.slice/crio-ec94561c9c8c3e787861c42e38c79c5d9d40e2503d790dbc16093708f1ba8929 WatchSource:0}: Error finding container ec94561c9c8c3e787861c42e38c79c5d9d40e2503d790dbc16093708f1ba8929: Status 404 returned error can't find the container with id ec94561c9c8c3e787861c42e38c79c5d9d40e2503d790dbc16093708f1ba8929
Oct 30 16:08:47.021674 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:47.021628    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2" event={"ID":"61ca2cb3-f6e9-402e-aba6-48a14871d95c","Type":"ContainerStarted","Data":"819cb7466336ff5303fd2f3e54d440c8d3477a3da94f793a8b962e7b248978d8"}
Oct 30 16:08:47.021674 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:47.021680    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2" event={"ID":"61ca2cb3-f6e9-402e-aba6-48a14871d95c","Type":"ContainerStarted","Data":"ec94561c9c8c3e787861c42e38c79c5d9d40e2503d790dbc16093708f1ba8929"}
Oct 30 16:08:47.064111 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:47.064029    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2" podStartSLOduration=2.064011189 podStartE2EDuration="2.064011189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:08:47.061972776 +0000 UTC m=+1836.497577124" watchObservedRunningTime="2026-10-30 16:08:47.064011189 +0000 UTC m=+1836.499615538"
Oct 30 16:08:47.961590 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:08:47.961553    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:08:48.027485 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:48.027452    4585 generic.go:356] "Generic (PLEG): container finished" podID="61ca2cb3-f6e9-402e-aba6-48a14871d95c" containerID="819cb7466336ff5303fd2f3e54d440c8d3477a3da94f793a8b962e7b248978d8" exitCode=0
Oct 30 16:08:48.027931 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:48.027532    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2" event={"ID":"61ca2cb3-f6e9-402e-aba6-48a14871d95c","Type":"ContainerDied","Data":"819cb7466336ff5303fd2f3e54d440c8d3477a3da94f793a8b962e7b248978d8"}
Oct 30 16:08:49.162326 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:49.162302    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:49.234837 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:49.234791    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-xks-ca\") pod \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") "
Oct 30 16:08:49.235041 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:49.234848    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61ca2cb3-f6e9-402e-aba6-48a14871d95c-kube-api-access-7sfhl\" (UniqueName: \"kubernetes.io/projected/61ca2cb3-f6e9-402e-aba6-48a14871d95c-kube-api-access-7sfhl\") pod \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") "
Oct 30 16:08:49.235041 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:49.234923    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-openshift-ca\") pod \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\" (UID: \"61ca2cb3-f6e9-402e-aba6-48a14871d95c\") "
Oct 30 16:08:49.235314 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:49.235292    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-openshift-ca" pod "61ca2cb3-f6e9-402e-aba6-48a14871d95c" (UID: "61ca2cb3-f6e9-402e-aba6-48a14871d95c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:08:49.236812 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:49.236788    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-xks-ca" pod "61ca2cb3-f6e9-402e-aba6-48a14871d95c" (UID: "61ca2cb3-f6e9-402e-aba6-48a14871d95c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:08:49.236925 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:49.236860    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61ca2cb3-f6e9-402e-aba6-48a14871d95c-kube-api-access-7sfhl" pod "61ca2cb3-f6e9-402e-aba6-48a14871d95c" (UID: "61ca2cb3-f6e9-402e-aba6-48a14871d95c"). InnerVolumeSpecName "kube-api-access-7sfhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:08:49.336027 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:49.335942    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:08:49.336027 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:49.335972    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/61ca2cb3-f6e9-402e-aba6-48a14871d95c-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:08:49.336027 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:49.335981    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sfhl\" (UniqueName: \"kubernetes.io/projected/61ca2cb3-f6e9-402e-aba6-48a14871d95c-kube-api-access-7sfhl\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:08:50.038027 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:50.037978    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2" event={"ID":"61ca2cb3-f6e9-402e-aba6-48a14871d95c","Type":"ContainerDied","Data":"ec94561c9c8c3e787861c42e38c79c5d9d40e2503d790dbc16093708f1ba8929"}
Oct 30 16:08:50.038289 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:50.038048    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec94561c9c8c3e787861c42e38c79c5d9d40e2503d790dbc16093708f1ba8929"
Oct 30 16:08:50.038289 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:50.038004    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"
Oct 30 16:08:57.865141 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.865063    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"]
Oct 30 16:08:57.865542 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.865455    4585 cpu_manager.go:414] "containerMap: removing container" podUID="61ca2cb3-f6e9-402e-aba6-48a14871d95c" containerName="revoke-keys"
Oct 30 16:08:57.865542 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.865468    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61ca2cb3-f6e9-402e-aba6-48a14871d95c" containerName="revoke-keys"
Oct 30 16:08:57.865542 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.865517    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="61ca2cb3-f6e9-402e-aba6-48a14871d95c" containerName="revoke-keys"
Oct 30 16:08:57.868421 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.868403    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:57.884067 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.884041    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-df488b-openshift-default-dockercfg-z98pp\""
Oct 30 16:08:57.897399 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.897371    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"]
Oct 30 16:08:57.917224 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.917195    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmgkm\" (UniqueName: \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-kube-api-access-mmgkm\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:57.917427 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.917240    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-credential-socket\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:57.917427 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.917341    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istiod-ca-cert\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:57.917427 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.917415    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-data\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:57.917561 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.917451    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-socket\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:57.917561 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.917496    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-podinfo\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:57.917561 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.917521    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-envoy\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:57.917667 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.917562    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-certs\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:57.917667 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:57.917582    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-token\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.018491 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.018456    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-credential-socket\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.018491 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.018497    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istiod-ca-cert\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.019010 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.018520    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-data\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.019010 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.018646    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-socket\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.019010 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.018714    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-podinfo\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.019010 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.018747    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-envoy\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.019010 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.018808    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-certs\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.019010 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.018841    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-token\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.019010 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.018937    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmgkm\" (UniqueName: \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-kube-api-access-mmgkm\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.019524 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.019024    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-data\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.019524 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.019231    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-credential-socket\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.019834 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.019785    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-certs\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.020031 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.020003    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istiod-ca-cert\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.020115 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.020070    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-socket\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.021510 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.021486    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-envoy\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.021612 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.021566    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-podinfo\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.042282 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.042247    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-token\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.042282 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.042260    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmgkm\" (UniqueName: \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-kube-api-access-mmgkm\") pod \"e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") " pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.180951 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.180909    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:58.498406 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.498373    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"]
Oct 30 16:08:58.499284 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:08:58.499252    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ede5d1_04cc_466a_8814_bd30e3f9e7d0.slice/crio-a72f6e49b60c49e000498d8ada1032b582dda52dc03c7713bbc1579941f01bdc WatchSource:0}: Error finding container a72f6e49b60c49e000498d8ada1032b582dda52dc03c7713bbc1579941f01bdc: Status 404 returned error can't find the container with id a72f6e49b60c49e000498d8ada1032b582dda52dc03c7713bbc1579941f01bdc
Oct 30 16:08:58.501844 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.501812    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:08:58.501945 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.501879    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:08:58.501945 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:58.501907    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:08:59.080644 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:59.080609    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" event={"ID":"85ede5d1-04cc-466a-8814-bd30e3f9e7d0","Type":"ContainerStarted","Data":"34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e"}
Oct 30 16:08:59.081049 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:59.080660    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" event={"ID":"85ede5d1-04cc-466a-8814-bd30e3f9e7d0","Type":"ContainerStarted","Data":"a72f6e49b60c49e000498d8ada1032b582dda52dc03c7713bbc1579941f01bdc"}
Oct 30 16:08:59.149150 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:59.149100    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" podStartSLOduration=2.149085374 podStartE2EDuration="2.149085374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:08:59.136655302 +0000 UTC m=+1848.572259650" watchObservedRunningTime="2026-10-30 16:08:59.149085374 +0000 UTC m=+1848.584689721"
Oct 30 16:08:59.181967 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:59.181930    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:08:59.186872 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:08:59.186846    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:09:00.084985 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:00.084956    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:09:00.086121 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:00.086093    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:09:03.159383 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.159328    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"]
Oct 30 16:09:03.163163 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.163142    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.193983 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.193941    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-df488b-serving-cert\""
Oct 30 16:09:03.193983 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.193945    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 16:09:03.194251 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.194039    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gp5lt\""
Oct 30 16:09:03.194251 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.194046    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 16:09:03.194251 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.194117    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 16:09:03.196549 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.196527    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"]
Oct 30 16:09:03.274426 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.274386    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q88wq\" (UniqueName: \"kubernetes.io/projected/f8bb4691-4084-4a4f-a265-2947e4895c25-kube-api-access-q88wq\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.274602 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.274506    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-metrics-certs\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.274602 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.274549    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.274602 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.274572    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8bb4691-4084-4a4f-a265-2947e4895c25-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.375941 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.375906    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q88wq\" (UniqueName: \"kubernetes.io/projected/f8bb4691-4084-4a4f-a265-2947e4895c25-kube-api-access-q88wq\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.376161 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.375965    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-metrics-certs\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.376161 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.375994    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.376161 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.376013    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8bb4691-4084-4a4f-a265-2947e4895c25-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.376868 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.376838    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8bb4691-4084-4a4f-a265-2947e4895c25-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.378603 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.378574    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.378723 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.378692    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-metrics-certs\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.400559 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.400532    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q88wq\" (UniqueName: \"kubernetes.io/projected/f8bb4691-4084-4a4f-a265-2947e4895c25-kube-api-access-q88wq\") pod \"maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.480489 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.480403    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:03.778505 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:03.778473    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"]
Oct 30 16:09:03.779094 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:09:03.779066    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8bb4691_4084_4a4f_a265_2947e4895c25.slice/crio-14731f67eead0c6538afc35937301bcfcf42c5f77e36a90162e3624aab9a8afd WatchSource:0}: Error finding container 14731f67eead0c6538afc35937301bcfcf42c5f77e36a90162e3624aab9a8afd: Status 404 returned error can't find the container with id 14731f67eead0c6538afc35937301bcfcf42c5f77e36a90162e3624aab9a8afd
Oct 30 16:09:04.104192 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:04.104102    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl" event={"ID":"f8bb4691-4084-4a4f-a265-2947e4895c25","Type":"ContainerStarted","Data":"14731f67eead0c6538afc35937301bcfcf42c5f77e36a90162e3624aab9a8afd"}
Oct 30 16:09:05.110416 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:05.110376    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl" event={"ID":"f8bb4691-4084-4a4f-a265-2947e4895c25","Type":"ContainerStarted","Data":"f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436"}
Oct 30 16:09:05.110951 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:05.110473    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:05.158103 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:05.158042    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl" podStartSLOduration=1.791583285 podStartE2EDuration="2.158022809s" totalImagesPullingTime="366.439524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:09:05.154004427 +0000 UTC m=+1854.589608777" watchObservedRunningTime="2026-10-30 16:09:05.158022809 +0000 UTC m=+1854.593627158"
Oct 30 16:09:11.120067 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:11.120037    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:09:15.549779 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:15.548840    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": read tcp 10.134.0.2:43268->10.134.0.93:15021: read: connection reset by peer" start-of-body=
Oct 30 16:09:15.549779 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:15.548905    4585 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": read tcp 10.134.0.2:43268->10.134.0.93:15021: read: connection reset by peer"
Oct 30 16:09:16.183095 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:16.183063    4585 generic.go:356] "Generic (PLEG): container finished" podID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerID="34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e" exitCode=0
Oct 30 16:09:16.183295 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:16.183115    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" event={"ID":"85ede5d1-04cc-466a-8814-bd30e3f9e7d0","Type":"ContainerDied","Data":"34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e"}
Oct 30 16:09:16.183531 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:16.183516    4585 scope.go:122] "RemoveContainer" containerID="34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e"
Oct 30 16:09:16.184683 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:16.184651    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:09:16.184746 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:16.184725    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:09:16.184789 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:16.184764    4585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Oct 30 16:09:17.190032 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:17.189998    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" event={"ID":"85ede5d1-04cc-466a-8814-bd30e3f9e7d0","Type":"ContainerStarted","Data":"b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a"}
Oct 30 16:09:17.982060 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:09:17.982018    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:09:18.181979 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:18.181945    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:09:18.182201 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:18.182154    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 16:09:18.182248 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:18.182194    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 16:09:19.181487 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:19.181448    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 16:09:19.181917 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:19.181503    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 16:09:20.182010 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:20.181963    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 16:09:20.182540 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:20.182029    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 16:09:21.182076 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:21.182036    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 16:09:21.182593 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:21.182095    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 16:09:22.182168 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:22.182122    4585 patch_prober.go:29] interesting pod/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 30 16:09:22.182614 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:22.182210    4585 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 30 16:09:23.184865 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:23.184834    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:09:23.185377 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:23.185129    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:09:23.185827 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:09:23.185808    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:09:47.960129 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:09:47.960094    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:10:02.835387 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:02.835325    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"]
Oct 30 16:10:02.835965 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:02.835614    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl" podUID="f8bb4691-4084-4a4f-a265-2947e4895c25" containerName="maas-api" containerID="cri-o://f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436" gracePeriod=30
Oct 30 16:10:04.086327 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.086296    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:10:04.135095 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.135060    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-metrics-certs\") pod \"f8bb4691-4084-4a4f-a265-2947e4895c25\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") "
Oct 30 16:10:04.135283 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.135118    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8bb4691-4084-4a4f-a265-2947e4895c25-kube-api-access-q88wq\" (UniqueName: \"kubernetes.io/projected/f8bb4691-4084-4a4f-a265-2947e4895c25-kube-api-access-q88wq\") pod \"f8bb4691-4084-4a4f-a265-2947e4895c25\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") "
Oct 30 16:10:04.135283 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.135143    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-maas-api-tls\") pod \"f8bb4691-4084-4a4f-a265-2947e4895c25\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") "
Oct 30 16:10:04.135283 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.135198    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8bb4691-4084-4a4f-a265-2947e4895c25-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8bb4691-4084-4a4f-a265-2947e4895c25-openshift-service-ca\") pod \"f8bb4691-4084-4a4f-a265-2947e4895c25\" (UID: \"f8bb4691-4084-4a4f-a265-2947e4895c25\") "
Oct 30 16:10:04.135860 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.135835    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8bb4691-4084-4a4f-a265-2947e4895c25-openshift-service-ca" pod "f8bb4691-4084-4a4f-a265-2947e4895c25" (UID: "f8bb4691-4084-4a4f-a265-2947e4895c25"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:10:04.137523 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.137492    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8bb4691-4084-4a4f-a265-2947e4895c25-kube-api-access-q88wq" pod "f8bb4691-4084-4a4f-a265-2947e4895c25" (UID: "f8bb4691-4084-4a4f-a265-2947e4895c25"). InnerVolumeSpecName "kube-api-access-q88wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:10:04.137940 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.137919    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-maas-api-tls" pod "f8bb4691-4084-4a4f-a265-2947e4895c25" (UID: "f8bb4691-4084-4a4f-a265-2947e4895c25"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:10:04.138036 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.138021    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-metrics-certs" pod "f8bb4691-4084-4a4f-a265-2947e4895c25" (UID: "f8bb4691-4084-4a4f-a265-2947e4895c25"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:10:04.237068 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.236982    4585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-metrics-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:04.237068 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.237012    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q88wq\" (UniqueName: \"kubernetes.io/projected/f8bb4691-4084-4a4f-a265-2947e4895c25-kube-api-access-q88wq\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:04.237068 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.237021    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8bb4691-4084-4a4f-a265-2947e4895c25-maas-api-tls\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:04.237068 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.237029    4585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8bb4691-4084-4a4f-a265-2947e4895c25-openshift-service-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:04.410573 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.408322    4585 generic.go:356] "Generic (PLEG): container finished" podID="f8bb4691-4084-4a4f-a265-2947e4895c25" containerID="f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436" exitCode=0
Oct 30 16:10:04.410573 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.408436    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl" event={"ID":"f8bb4691-4084-4a4f-a265-2947e4895c25","Type":"ContainerDied","Data":"f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436"}
Oct 30 16:10:04.410573 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.408487    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl" event={"ID":"f8bb4691-4084-4a4f-a265-2947e4895c25","Type":"ContainerDied","Data":"14731f67eead0c6538afc35937301bcfcf42c5f77e36a90162e3624aab9a8afd"}
Oct 30 16:10:04.410573 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.408516    4585 scope.go:122] "RemoveContainer" containerID="f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436"
Oct 30 16:10:04.410573 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.408672    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"
Oct 30 16:10:04.421373 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.421323    4585 scope.go:122] "RemoveContainer" containerID="f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436"
Oct 30 16:10:04.421801 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:10:04.421778    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436\": container with ID starting with f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436 not found: ID does not exist" containerID="f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436"
Oct 30 16:10:04.421889 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.421822    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436"} err="failed to get container status \"f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436\": rpc error: code = NotFound desc = could not find container \"f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436\": container with ID starting with f9c0441986c65e0a5b164106cf7d5f75decd7fb4d6679760b951c6cdf59d2436 not found: ID does not exist"
Oct 30 16:10:04.461827 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.461792    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"]
Oct 30 16:10:04.473775 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.473744    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-df488b-7cd7566697-blxtl"]
Oct 30 16:10:04.700805 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:04.700770    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8bb4691-4084-4a4f-a265-2947e4895c25" path="/var/lib/kubelet/pods/f8bb4691-4084-4a4f-a265-2947e4895c25/volumes"
Oct 30 16:10:11.049856 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:11.049820    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"]
Oct 30 16:10:11.050707 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:11.050651    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy" containerID="cri-o://b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a" gracePeriod=30
Oct 30 16:10:16.303165 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.303133    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:10:16.360171 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.359283    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-envoy\") pod \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") "
Oct 30 16:10:16.360171 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.359330    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-token\" (UniqueName: \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-token\") pod \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") "
Oct 30 16:10:16.360171 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.359491    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-data\") pod \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") "
Oct 30 16:10:16.360171 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.359560    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-podinfo\") pod \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") "
Oct 30 16:10:16.360171 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.359654    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istiod-ca-cert\") pod \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") "
Oct 30 16:10:16.360171 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.359690    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-certs\") pod \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") "
Oct 30 16:10:16.360171 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.359732    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-socket\") pod \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") "
Oct 30 16:10:16.360171 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.359762    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-kube-api-access-mmgkm\" (UniqueName: \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-kube-api-access-mmgkm\") pod \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") "
Oct 30 16:10:16.360171 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.359804    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-credential-socket\") pod \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\" (UID: \"85ede5d1-04cc-466a-8814-bd30e3f9e7d0\") "
Oct 30 16:10:16.361504 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.360342    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-credential-socket" pod "85ede5d1-04cc-466a-8814-bd30e3f9e7d0" (UID: "85ede5d1-04cc-466a-8814-bd30e3f9e7d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:10:16.361504 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.360538    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-data" pod "85ede5d1-04cc-466a-8814-bd30e3f9e7d0" (UID: "85ede5d1-04cc-466a-8814-bd30e3f9e7d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:10:16.361504 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.361166    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-certs" pod "85ede5d1-04cc-466a-8814-bd30e3f9e7d0" (UID: "85ede5d1-04cc-466a-8814-bd30e3f9e7d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:10:16.361504 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.361459    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istiod-ca-cert" pod "85ede5d1-04cc-466a-8814-bd30e3f9e7d0" (UID: "85ede5d1-04cc-466a-8814-bd30e3f9e7d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:10:16.361726 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.361694    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-socket" pod "85ede5d1-04cc-466a-8814-bd30e3f9e7d0" (UID: "85ede5d1-04cc-466a-8814-bd30e3f9e7d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:10:16.365577 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.365545    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-envoy" pod "85ede5d1-04cc-466a-8814-bd30e3f9e7d0" (UID: "85ede5d1-04cc-466a-8814-bd30e3f9e7d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:10:16.365686 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.365607    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-podinfo" pod "85ede5d1-04cc-466a-8814-bd30e3f9e7d0" (UID: "85ede5d1-04cc-466a-8814-bd30e3f9e7d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 16:10:16.365686 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.365613    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-kube-api-access-mmgkm" pod "85ede5d1-04cc-466a-8814-bd30e3f9e7d0" (UID: "85ede5d1-04cc-466a-8814-bd30e3f9e7d0"). InnerVolumeSpecName "kube-api-access-mmgkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:10:16.365686 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.365613    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-token" pod "85ede5d1-04cc-466a-8814-bd30e3f9e7d0" (UID: "85ede5d1-04cc-466a-8814-bd30e3f9e7d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:10:16.460621 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.460592    4585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-envoy\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:16.460621 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.460617    4585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-token\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:16.460621 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.460626    4585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-data\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:16.460852 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.460634    4585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istio-podinfo\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:16.460852 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.460642    4585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-istiod-ca-cert\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:16.460852 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.460650    4585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:16.460852 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.460657    4585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-workload-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:16.460852 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.460665    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmgkm\" (UniqueName: \"kubernetes.io/projected/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-kube-api-access-mmgkm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:16.460852 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.460675    4585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85ede5d1-04cc-466a-8814-bd30e3f9e7d0-credential-socket\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:16.468976 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.468947    4585 generic.go:356] "Generic (PLEG): container finished" podID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerID="b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a" exitCode=0
Oct 30 16:10:16.469149 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.469015    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"
Oct 30 16:10:16.469149 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.469035    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" event={"ID":"85ede5d1-04cc-466a-8814-bd30e3f9e7d0","Type":"ContainerDied","Data":"b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a"}
Oct 30 16:10:16.469149 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.469100    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj" event={"ID":"85ede5d1-04cc-466a-8814-bd30e3f9e7d0","Type":"ContainerDied","Data":"a72f6e49b60c49e000498d8ada1032b582dda52dc03c7713bbc1579941f01bdc"}
Oct 30 16:10:16.469149 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.469131    4585 scope.go:122] "RemoveContainer" containerID="b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a"
Oct 30 16:10:16.478639 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.478603    4585 scope.go:122] "RemoveContainer" containerID="34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e"
Oct 30 16:10:16.486360 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.486328    4585 scope.go:122] "RemoveContainer" containerID="b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a"
Oct 30 16:10:16.486625 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:10:16.486605    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a\": container with ID starting with b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a not found: ID does not exist" containerID="b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a"
Oct 30 16:10:16.486666 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.486633    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a"} err="failed to get container status \"b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a\": rpc error: code = NotFound desc = could not find container \"b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a\": container with ID starting with b08b8b1e33ebf359b7752041fa7e48016916ae8879ab1f074f9fc4e6f33b3e0a not found: ID does not exist"
Oct 30 16:10:16.486666 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.486648    4585 scope.go:122] "RemoveContainer" containerID="34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e"
Oct 30 16:10:16.486894 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:10:16.486875    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e\": container with ID starting with 34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e not found: ID does not exist" containerID="34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e"
Oct 30 16:10:16.486929 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.486902    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e"} err="failed to get container status \"34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e\": rpc error: code = NotFound desc = could not find container \"34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e\": container with ID starting with 34838eb97fa7955f93ae12ab2ea6b724e952992b0b45b31a325ec2cdd153792e not found: ID does not exist"
Oct 30 16:10:16.520200 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.520165    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"]
Oct 30 16:10:16.524196 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.524169    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-df488b-openshift-default-55667d9c8fd48zj"]
Oct 30 16:10:16.700807 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:16.700774    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" path="/var/lib/kubelet/pods/85ede5d1-04cc-466a-8814-bd30e3f9e7d0/volumes"
Oct 30 16:10:17.976634 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:10:17.976594    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:10:33.152847 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.152753    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"]
Oct 30 16:10:33.153406 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.153281    4585 cpu_manager.go:414] "containerMap: removing container" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy"
Oct 30 16:10:33.153406 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.153301    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy"
Oct 30 16:10:33.153406 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.153318    4585 cpu_manager.go:414] "containerMap: removing container" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy"
Oct 30 16:10:33.153406 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.153326    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy"
Oct 30 16:10:33.153406 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.153368    4585 cpu_manager.go:414] "containerMap: removing container" podUID="f8bb4691-4084-4a4f-a265-2947e4895c25" containerName="maas-api"
Oct 30 16:10:33.153406 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.153376    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8bb4691-4084-4a4f-a265-2947e4895c25" containerName="maas-api"
Oct 30 16:10:33.153740 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.153492    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy"
Oct 30 16:10:33.153740 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.153504    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="85ede5d1-04cc-466a-8814-bd30e3f9e7d0" containerName="istio-proxy"
Oct 30 16:10:33.153740 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.153518    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8bb4691-4084-4a4f-a265-2947e4895c25" containerName="maas-api"
Oct 30 16:10:33.157481 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.157459    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.178555 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.178516    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"]
Oct 30 16:10:33.187917 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.187892    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-3c05ee64-kserve-self-signed-certs\""
Oct 30 16:10:33.281246 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.281207    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tls-certs\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.281441 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.281256    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-home\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.281441 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.281332    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm6z6\" (UniqueName: \"kubernetes.io/projected/6f35dfec-02d1-40d1-be21-7152bd6dc97b-kube-api-access-rm6z6\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.281441 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.281384    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tmp-dir\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.281441 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.281436    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-model-cache\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.281590 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.281527    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-dshm\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.382372 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.382307    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tls-certs\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.382590 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.382392    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-home\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.382590 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.382433    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm6z6\" (UniqueName: \"kubernetes.io/projected/6f35dfec-02d1-40d1-be21-7152bd6dc97b-kube-api-access-rm6z6\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.382590 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.382473    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tmp-dir\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.382590 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.382542    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-model-cache\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.382590 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.382583    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-dshm\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.382926 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.382896    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-home\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.383241 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.383213    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tmp-dir\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.383390 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.383303    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-model-cache\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.385843 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.385815    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-dshm\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.386219 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.386199    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tls-certs\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.406552 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.406485    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm6z6\" (UniqueName: \"kubernetes.io/projected/6f35dfec-02d1-40d1-be21-7152bd6dc97b-kube-api-access-rm6z6\") pod \"e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") " pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.470204 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.470169    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:33.770648 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:10:33.770614    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f35dfec_02d1_40d1_be21_7152bd6dc97b.slice/crio-d3f90967479bd46a166acaf52c911ba1b56a77874ba48f3bad30bf7aa766ebcf WatchSource:0}: Error finding container d3f90967479bd46a166acaf52c911ba1b56a77874ba48f3bad30bf7aa766ebcf: Status 404 returned error can't find the container with id d3f90967479bd46a166acaf52c911ba1b56a77874ba48f3bad30bf7aa766ebcf
Oct 30 16:10:33.770648 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:33.770622    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"]
Oct 30 16:10:34.574597 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:34.574557    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp" event={"ID":"6f35dfec-02d1-40d1-be21-7152bd6dc97b","Type":"ContainerStarted","Data":"35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1"}
Oct 30 16:10:34.574597 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:34.574602    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp" event={"ID":"6f35dfec-02d1-40d1-be21-7152bd6dc97b","Type":"ContainerStarted","Data":"d3f90967479bd46a166acaf52c911ba1b56a77874ba48f3bad30bf7aa766ebcf"}
Oct 30 16:10:34.617722 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:34.617670    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp" podStartSLOduration=1.617654707 podStartE2EDuration="1.617654707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:10:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:10:34.61589827 +0000 UTC m=+1944.051502619" watchObservedRunningTime="2026-10-30 16:10:34.617654707 +0000 UTC m=+1944.053259095"
Oct 30 16:10:35.471018 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:35.470983    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:43.471049 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:43.470998    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:43.483547 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:43.483523    4585 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:43.638517 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:43.638487    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:10:47.959507 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:10:47.959473    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:10:51.542097 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.542049    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"]
Oct 30 16:10:51.546209 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.546176    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:51.567769 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.563443    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 16:10:51.567769 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.563536    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 16:10:51.580406 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.575463    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 16:10:51.580406 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.575823    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 16:10:51.580406 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.579343    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"]
Oct 30 16:10:51.597161 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.597121    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/519752e8-6e80-476c-81b9-3811bd540663-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-67d7r\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:51.597383 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.597277    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs2lj\" (UniqueName: \"kubernetes.io/projected/519752e8-6e80-476c-81b9-3811bd540663-kube-api-access-gs2lj\") pod \"maas-api-revoke-sub-452b6d7b-67d7r\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:51.597455 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.597420    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/519752e8-6e80-476c-81b9-3811bd540663-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-67d7r\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:51.698860 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.698815    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/519752e8-6e80-476c-81b9-3811bd540663-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-67d7r\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:51.699063 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.698915    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs2lj\" (UniqueName: \"kubernetes.io/projected/519752e8-6e80-476c-81b9-3811bd540663-kube-api-access-gs2lj\") pod \"maas-api-revoke-sub-452b6d7b-67d7r\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:51.699063 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.698979    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/519752e8-6e80-476c-81b9-3811bd540663-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-67d7r\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:51.699866 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.699833    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/519752e8-6e80-476c-81b9-3811bd540663-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-67d7r\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:51.702228 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.702199    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/519752e8-6e80-476c-81b9-3811bd540663-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-67d7r\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:51.724181 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.724142    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs2lj\" (UniqueName: \"kubernetes.io/projected/519752e8-6e80-476c-81b9-3811bd540663-kube-api-access-gs2lj\") pod \"maas-api-revoke-sub-452b6d7b-67d7r\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:51.867234 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:51.867125    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:52.204897 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:52.204504    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"]
Oct 30 16:10:52.212398 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:10:52.212367    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519752e8_6e80_476c_81b9_3811bd540663.slice/crio-f0986a6476c029f4e9a3a8fa6489ada7fb06957593b1433c6d709881a07d6d3b WatchSource:0}: Error finding container f0986a6476c029f4e9a3a8fa6489ada7fb06957593b1433c6d709881a07d6d3b: Status 404 returned error can't find the container with id f0986a6476c029f4e9a3a8fa6489ada7fb06957593b1433c6d709881a07d6d3b
Oct 30 16:10:52.685537 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:52.685487    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r" event={"ID":"519752e8-6e80-476c-81b9-3811bd540663","Type":"ContainerStarted","Data":"f0986a6476c029f4e9a3a8fa6489ada7fb06957593b1433c6d709881a07d6d3b"}
Oct 30 16:10:53.707369 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:53.707208    4585 generic.go:356] "Generic (PLEG): container finished" podID="519752e8-6e80-476c-81b9-3811bd540663" containerID="2e66ac86ba404d98c23c1ef118a79b32dbfbb7628b24a08a199a1b54e3414097" exitCode=0
Oct 30 16:10:53.707369 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:53.707323    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r" event={"ID":"519752e8-6e80-476c-81b9-3811bd540663","Type":"ContainerDied","Data":"2e66ac86ba404d98c23c1ef118a79b32dbfbb7628b24a08a199a1b54e3414097"}
Oct 30 16:10:54.842068 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:54.842039    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:54.974765 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:54.974660    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/519752e8-6e80-476c-81b9-3811bd540663-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/519752e8-6e80-476c-81b9-3811bd540663-maas-api-openshift-ca\") pod \"519752e8-6e80-476c-81b9-3811bd540663\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") "
Oct 30 16:10:54.974950 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:54.974785    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/519752e8-6e80-476c-81b9-3811bd540663-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/519752e8-6e80-476c-81b9-3811bd540663-maas-api-xks-ca\") pod \"519752e8-6e80-476c-81b9-3811bd540663\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") "
Oct 30 16:10:54.974950 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:54.974901    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/519752e8-6e80-476c-81b9-3811bd540663-kube-api-access-gs2lj\" (UniqueName: \"kubernetes.io/projected/519752e8-6e80-476c-81b9-3811bd540663-kube-api-access-gs2lj\") pod \"519752e8-6e80-476c-81b9-3811bd540663\" (UID: \"519752e8-6e80-476c-81b9-3811bd540663\") "
Oct 30 16:10:54.975061 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:54.975025    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/519752e8-6e80-476c-81b9-3811bd540663-maas-api-openshift-ca" pod "519752e8-6e80-476c-81b9-3811bd540663" (UID: "519752e8-6e80-476c-81b9-3811bd540663"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:10:54.975212 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:54.975195    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/519752e8-6e80-476c-81b9-3811bd540663-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:54.977021 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:54.976987    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/519752e8-6e80-476c-81b9-3811bd540663-maas-api-xks-ca" pod "519752e8-6e80-476c-81b9-3811bd540663" (UID: "519752e8-6e80-476c-81b9-3811bd540663"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:10:54.977176 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:54.977031    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519752e8-6e80-476c-81b9-3811bd540663-kube-api-access-gs2lj" pod "519752e8-6e80-476c-81b9-3811bd540663" (UID: "519752e8-6e80-476c-81b9-3811bd540663"). InnerVolumeSpecName "kube-api-access-gs2lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:10:55.076571 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:55.076537    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs2lj\" (UniqueName: \"kubernetes.io/projected/519752e8-6e80-476c-81b9-3811bd540663-kube-api-access-gs2lj\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:55.076571 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:55.076568    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/519752e8-6e80-476c-81b9-3811bd540663-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:10:55.716683 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:55.716644    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r" event={"ID":"519752e8-6e80-476c-81b9-3811bd540663","Type":"ContainerDied","Data":"f0986a6476c029f4e9a3a8fa6489ada7fb06957593b1433c6d709881a07d6d3b"}
Oct 30 16:10:55.716683 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:55.716677    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-67d7r"
Oct 30 16:10:55.716913 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:10:55.716701    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0986a6476c029f4e9a3a8fa6489ada7fb06957593b1433c6d709881a07d6d3b"
Oct 30 16:11:05.855810 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:05.855596    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"]
Oct 30 16:11:09.649340 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.649289    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"]
Oct 30 16:11:09.649894 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.649866    4585 cpu_manager.go:414] "containerMap: removing container" podUID="519752e8-6e80-476c-81b9-3811bd540663" containerName="revoke-keys"
Oct 30 16:11:09.649894 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.649886    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="519752e8-6e80-476c-81b9-3811bd540663" containerName="revoke-keys"
Oct 30 16:11:09.650042 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.649973    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="519752e8-6e80-476c-81b9-3811bd540663" containerName="revoke-keys"
Oct 30 16:11:09.653535 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.653513    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:09.670434 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.670399    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"]
Oct 30 16:11:09.680654 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.680625    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 16:11:09.680822 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.680710    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 16:11:09.680822 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.680761    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 16:11:09.682871 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.682837    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 16:11:09.734071 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.734032    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-hzpmw\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:09.734274 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.734115    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcv78\" (UniqueName: \"kubernetes.io/projected/6a7099d5-e741-4216-8774-e5dac6d82f81-kube-api-access-fcv78\") pod \"maas-api-revoke-sub-3e31a506-hzpmw\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:09.734274 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.734249    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-hzpmw\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:09.835058 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.835011    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-hzpmw\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:09.835261 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.835147    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-hzpmw\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:09.835261 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.835209    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcv78\" (UniqueName: \"kubernetes.io/projected/6a7099d5-e741-4216-8774-e5dac6d82f81-kube-api-access-fcv78\") pod \"maas-api-revoke-sub-3e31a506-hzpmw\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:09.835823 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.835791    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-hzpmw\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:09.837743 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.837717    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-hzpmw\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:09.859418 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.859387    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcv78\" (UniqueName: \"kubernetes.io/projected/6a7099d5-e741-4216-8774-e5dac6d82f81-kube-api-access-fcv78\") pod \"maas-api-revoke-sub-3e31a506-hzpmw\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:09.978461 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:09.978342    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:10.324574 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:10.324527    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"]
Oct 30 16:11:10.822431 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:10.822330    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw" event={"ID":"6a7099d5-e741-4216-8774-e5dac6d82f81","Type":"ContainerStarted","Data":"cdbe3b936ead4a3da2f2ca8b2b96d6ddeeb29caffb09f31d5ef086a6ce804f9f"}
Oct 30 16:11:11.828378 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:11.828169    4585 generic.go:356] "Generic (PLEG): container finished" podID="6a7099d5-e741-4216-8774-e5dac6d82f81" containerID="d7bbc13f06b48f1d03b4913f861cd49ad1279b5299a4a518daa9883597bf5fcf" exitCode=0
Oct 30 16:11:11.828378 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:11.828280    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw" event={"ID":"6a7099d5-e741-4216-8774-e5dac6d82f81","Type":"ContainerDied","Data":"d7bbc13f06b48f1d03b4913f861cd49ad1279b5299a4a518daa9883597bf5fcf"}
Oct 30 16:11:12.076963 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:12.076931    4585 scope.go:122] "RemoveContainer" containerID="e1ffce583acd3a0e0f77268985dc08c97906c179e665534f26c502927c780ef3"
Oct 30 16:11:12.965069 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:12.965040    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:13.036461 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.036426    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a7099d5-e741-4216-8774-e5dac6d82f81-kube-api-access-fcv78\" (UniqueName: \"kubernetes.io/projected/6a7099d5-e741-4216-8774-e5dac6d82f81-kube-api-access-fcv78\") pod \"6a7099d5-e741-4216-8774-e5dac6d82f81\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") "
Oct 30 16:11:13.036461 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.036465    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-xks-ca\") pod \"6a7099d5-e741-4216-8774-e5dac6d82f81\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") "
Oct 30 16:11:13.036714 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.036553    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-openshift-ca\") pod \"6a7099d5-e741-4216-8774-e5dac6d82f81\" (UID: \"6a7099d5-e741-4216-8774-e5dac6d82f81\") "
Oct 30 16:11:13.036957 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.036924    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-openshift-ca" pod "6a7099d5-e741-4216-8774-e5dac6d82f81" (UID: "6a7099d5-e741-4216-8774-e5dac6d82f81"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:11:13.038738 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.038712    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-xks-ca" pod "6a7099d5-e741-4216-8774-e5dac6d82f81" (UID: "6a7099d5-e741-4216-8774-e5dac6d82f81"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:11:13.038871 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.038850    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a7099d5-e741-4216-8774-e5dac6d82f81-kube-api-access-fcv78" pod "6a7099d5-e741-4216-8774-e5dac6d82f81" (UID: "6a7099d5-e741-4216-8774-e5dac6d82f81"). InnerVolumeSpecName "kube-api-access-fcv78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:11:13.137483 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.137385    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:13.137483 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.137418    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcv78\" (UniqueName: \"kubernetes.io/projected/6a7099d5-e741-4216-8774-e5dac6d82f81-kube-api-access-fcv78\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:13.137483 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.137429    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6a7099d5-e741-4216-8774-e5dac6d82f81-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:13.838430 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.838399    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw"
Oct 30 16:11:13.838634 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.838390    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-hzpmw" event={"ID":"6a7099d5-e741-4216-8774-e5dac6d82f81","Type":"ContainerDied","Data":"cdbe3b936ead4a3da2f2ca8b2b96d6ddeeb29caffb09f31d5ef086a6ce804f9f"}
Oct 30 16:11:13.838634 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:13.838516    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdbe3b936ead4a3da2f2ca8b2b96d6ddeeb29caffb09f31d5ef086a6ce804f9f"
Oct 30 16:11:17.962428 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:11:17.962391    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:11:20.889609 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:20.889570    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp" podUID="6f35dfec-02d1-40d1-be21-7152bd6dc97b" containerName="main" containerID="cri-o://35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1" gracePeriod=45
Oct 30 16:11:21.148911 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.148882    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:11:21.219161 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219128    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f35dfec-02d1-40d1-be21-7152bd6dc97b-kube-api-access-rm6z6\" (UniqueName: \"kubernetes.io/projected/6f35dfec-02d1-40d1-be21-7152bd6dc97b-kube-api-access-rm6z6\") pod \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") "
Oct 30 16:11:21.219365 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219327    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tls-certs\") pod \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") "
Oct 30 16:11:21.219431 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219415    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-dshm\") pod \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") "
Oct 30 16:11:21.219472 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219450    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-model-cache\") pod \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") "
Oct 30 16:11:21.219539 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219525    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tmp-dir\") pod \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") "
Oct 30 16:11:21.219592 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219556    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-home\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-home\") pod \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\" (UID: \"6f35dfec-02d1-40d1-be21-7152bd6dc97b\") "
Oct 30 16:11:21.219788 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219757    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-model-cache" pod "6f35dfec-02d1-40d1-be21-7152bd6dc97b" (UID: "6f35dfec-02d1-40d1-be21-7152bd6dc97b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:11:21.219923 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219793    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tmp-dir" pod "6f35dfec-02d1-40d1-be21-7152bd6dc97b" (UID: "6f35dfec-02d1-40d1-be21-7152bd6dc97b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:11:21.220009 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219953    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-home" pod "6f35dfec-02d1-40d1-be21-7152bd6dc97b" (UID: "6f35dfec-02d1-40d1-be21-7152bd6dc97b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:11:21.220009 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219976    4585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-model-cache\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:21.220009 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.219999    4585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tmp-dir\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:21.221638 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.221611    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-dshm" pod "6f35dfec-02d1-40d1-be21-7152bd6dc97b" (UID: "6f35dfec-02d1-40d1-be21-7152bd6dc97b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 16:11:21.221847 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.221824    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f35dfec-02d1-40d1-be21-7152bd6dc97b-kube-api-access-rm6z6" pod "6f35dfec-02d1-40d1-be21-7152bd6dc97b" (UID: "6f35dfec-02d1-40d1-be21-7152bd6dc97b"). InnerVolumeSpecName "kube-api-access-rm6z6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:11:21.221952 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.221827    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tls-certs" pod "6f35dfec-02d1-40d1-be21-7152bd6dc97b" (UID: "6f35dfec-02d1-40d1-be21-7152bd6dc97b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:11:21.320930 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.320891    4585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-home\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:21.320930 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.320932    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm6z6\" (UniqueName: \"kubernetes.io/projected/6f35dfec-02d1-40d1-be21-7152bd6dc97b-kube-api-access-rm6z6\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:21.321189 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.320949    4585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f35dfec-02d1-40d1-be21-7152bd6dc97b-tls-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:21.321189 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.320962    4585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f35dfec-02d1-40d1-be21-7152bd6dc97b-dshm\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:21.889372 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.888657    4585 generic.go:356] "Generic (PLEG): container finished" podID="6f35dfec-02d1-40d1-be21-7152bd6dc97b" containerID="35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1" exitCode=0
Oct 30 16:11:21.889372 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.888717    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp" event={"ID":"6f35dfec-02d1-40d1-be21-7152bd6dc97b","Type":"ContainerDied","Data":"35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1"}
Oct 30 16:11:21.889372 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.888757    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp" event={"ID":"6f35dfec-02d1-40d1-be21-7152bd6dc97b","Type":"ContainerDied","Data":"d3f90967479bd46a166acaf52c911ba1b56a77874ba48f3bad30bf7aa766ebcf"}
Oct 30 16:11:21.889372 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.888783    4585 scope.go:122] "RemoveContainer" containerID="35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1"
Oct 30 16:11:21.889372 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.888945    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"
Oct 30 16:11:21.918488 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.918461    4585 scope.go:122] "RemoveContainer" containerID="35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1"
Oct 30 16:11:21.918922 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:11:21.918883    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1\": container with ID starting with 35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1 not found: ID does not exist" containerID="35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1"
Oct 30 16:11:21.919055 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.918936    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1"} err="failed to get container status \"35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1\": rpc error: code = NotFound desc = could not find container \"35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1\": container with ID starting with 35ab1401248b507b385e9e4fd83eb5394c5ca4a266b8ecccfa75f9ba36a78ff1 not found: ID does not exist"
Oct 30 16:11:21.969372 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.954669    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"]
Oct 30 16:11:21.969372 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:21.968746    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-3c05ee64-kserve-54c9bc57df-7jmpp"]
Oct 30 16:11:22.707615 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:22.707538    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f35dfec-02d1-40d1-be21-7152bd6dc97b" path="/var/lib/kubelet/pods/6f35dfec-02d1-40d1-be21-7152bd6dc97b/volumes"
Oct 30 16:11:33.352415 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.352375    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"]
Oct 30 16:11:33.352835 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.352736    4585 cpu_manager.go:414] "containerMap: removing container" podUID="6a7099d5-e741-4216-8774-e5dac6d82f81" containerName="revoke-keys"
Oct 30 16:11:33.352835 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.352748    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a7099d5-e741-4216-8774-e5dac6d82f81" containerName="revoke-keys"
Oct 30 16:11:33.352835 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.352774    4585 cpu_manager.go:414] "containerMap: removing container" podUID="6f35dfec-02d1-40d1-be21-7152bd6dc97b" containerName="main"
Oct 30 16:11:33.352835 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.352779    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f35dfec-02d1-40d1-be21-7152bd6dc97b" containerName="main"
Oct 30 16:11:33.352983 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.352842    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f35dfec-02d1-40d1-be21-7152bd6dc97b" containerName="main"
Oct 30 16:11:33.352983 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.352852    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a7099d5-e741-4216-8774-e5dac6d82f81" containerName="revoke-keys"
Oct 30 16:11:33.356867 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.356846    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.377520 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.377494    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"]
Oct 30 16:11:33.386870 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.386843    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 16:11:33.387031 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.386936    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 16:11:33.387031 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.386986    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 16:11:33.387180 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.387062    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wsxv7\""
Oct 30 16:11:33.494823 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.494787    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0567114c-b4jw4\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.495005 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.494834    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0567114c-b4jw4\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.495005 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.494965    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvklf\" (UniqueName: \"kubernetes.io/projected/cfb55879-1ae9-4f3a-930a-9d970559fa28-kube-api-access-jvklf\") pod \"maas-api-revoke-sub-0567114c-b4jw4\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.596485 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.596452    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0567114c-b4jw4\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.596664 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.596495    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0567114c-b4jw4\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.596664 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.596557    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvklf\" (UniqueName: \"kubernetes.io/projected/cfb55879-1ae9-4f3a-930a-9d970559fa28-kube-api-access-jvklf\") pod \"maas-api-revoke-sub-0567114c-b4jw4\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.597193 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.597160    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0567114c-b4jw4\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.598814 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.598791    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0567114c-b4jw4\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.620217 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.620147    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvklf\" (UniqueName: \"kubernetes.io/projected/cfb55879-1ae9-4f3a-930a-9d970559fa28-kube-api-access-jvklf\") pod \"maas-api-revoke-sub-0567114c-b4jw4\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.670147 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.670117    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:33.965845 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:33.965819    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"]
Oct 30 16:11:33.966645 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:11:33.966615    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfb55879_1ae9_4f3a_930a_9d970559fa28.slice/crio-9bb41b5f9689bbd6513eeac650899c787cb0ac6693d290bc7e1d17e78d6ee9e2 WatchSource:0}: Error finding container 9bb41b5f9689bbd6513eeac650899c787cb0ac6693d290bc7e1d17e78d6ee9e2: Status 404 returned error can't find the container with id 9bb41b5f9689bbd6513eeac650899c787cb0ac6693d290bc7e1d17e78d6ee9e2
Oct 30 16:11:34.964575 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:34.964536    4585 generic.go:356] "Generic (PLEG): container finished" podID="cfb55879-1ae9-4f3a-930a-9d970559fa28" containerID="1e92e9d176c4ac12e2a1a55cecca7010d29b32691af6a43ce831e6f607c181c9" exitCode=0
Oct 30 16:11:34.964960 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:34.964625    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4" event={"ID":"cfb55879-1ae9-4f3a-930a-9d970559fa28","Type":"ContainerDied","Data":"1e92e9d176c4ac12e2a1a55cecca7010d29b32691af6a43ce831e6f607c181c9"}
Oct 30 16:11:34.964960 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:34.964675    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4" event={"ID":"cfb55879-1ae9-4f3a-930a-9d970559fa28","Type":"ContainerStarted","Data":"9bb41b5f9689bbd6513eeac650899c787cb0ac6693d290bc7e1d17e78d6ee9e2"}
Oct 30 16:11:36.111135 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.111108    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:36.224287 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.224205    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-xks-ca\") pod \"cfb55879-1ae9-4f3a-930a-9d970559fa28\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") "
Oct 30 16:11:36.224465 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.224303    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-openshift-ca\") pod \"cfb55879-1ae9-4f3a-930a-9d970559fa28\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") "
Oct 30 16:11:36.224465 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.224374    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfb55879-1ae9-4f3a-930a-9d970559fa28-kube-api-access-jvklf\" (UniqueName: \"kubernetes.io/projected/cfb55879-1ae9-4f3a-930a-9d970559fa28-kube-api-access-jvklf\") pod \"cfb55879-1ae9-4f3a-930a-9d970559fa28\" (UID: \"cfb55879-1ae9-4f3a-930a-9d970559fa28\") "
Oct 30 16:11:36.224703 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.224674    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-openshift-ca" pod "cfb55879-1ae9-4f3a-930a-9d970559fa28" (UID: "cfb55879-1ae9-4f3a-930a-9d970559fa28"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:11:36.226458 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.226427    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb55879-1ae9-4f3a-930a-9d970559fa28-kube-api-access-jvklf" pod "cfb55879-1ae9-4f3a-930a-9d970559fa28" (UID: "cfb55879-1ae9-4f3a-930a-9d970559fa28"). InnerVolumeSpecName "kube-api-access-jvklf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:11:36.226554 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.226508    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-xks-ca" pod "cfb55879-1ae9-4f3a-930a-9d970559fa28" (UID: "cfb55879-1ae9-4f3a-930a-9d970559fa28"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:11:36.325819 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.325783    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvklf\" (UniqueName: \"kubernetes.io/projected/cfb55879-1ae9-4f3a-930a-9d970559fa28-kube-api-access-jvklf\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:36.325819 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.325815    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-xks-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:36.325819 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.325825    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cfb55879-1ae9-4f3a-930a-9d970559fa28-maas-api-openshift-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:36.975940 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.975835    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4" event={"ID":"cfb55879-1ae9-4f3a-930a-9d970559fa28","Type":"ContainerDied","Data":"9bb41b5f9689bbd6513eeac650899c787cb0ac6693d290bc7e1d17e78d6ee9e2"}
Oct 30 16:11:36.975940 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.975863    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0567114c-b4jw4"
Oct 30 16:11:36.975940 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:36.975897    4585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9bb41b5f9689bbd6513eeac650899c787cb0ac6693d290bc7e1d17e78d6ee9e2"
Oct 30 16:11:43.028535 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:43.028491    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"]
Oct 30 16:11:43.040865 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:43.040837    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-td9dz"]
Oct 30 16:11:44.700453 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:44.700422    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2bf107f-ff92-4694-9cb2-409bd37eaa89" path="/var/lib/kubelet/pods/c2bf107f-ff92-4694-9cb2-409bd37eaa89/volumes"
Oct 30 16:11:45.878009 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.877968    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"]
Oct 30 16:11:45.878418 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.878367    4585 cpu_manager.go:414] "containerMap: removing container" podUID="cfb55879-1ae9-4f3a-930a-9d970559fa28" containerName="revoke-keys"
Oct 30 16:11:45.878418 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.878381    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfb55879-1ae9-4f3a-930a-9d970559fa28" containerName="revoke-keys"
Oct 30 16:11:45.878497 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.878444    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfb55879-1ae9-4f3a-930a-9d970559fa28" containerName="revoke-keys"
Oct 30 16:11:45.882749 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.882731    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:45.896159 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.896130    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"]
Oct 30 16:11:45.900062 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.900042    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:45.905807 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.905773    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 16:11:45.905923 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.905805    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 16:11:45.912256 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.912235    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 16:11:45.912392 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.912300    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 16:11:45.912392 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.912335    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gp5lt\""
Oct 30 16:11:45.917006 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.916987    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-metrics-certs\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:45.917071 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.917018    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4977e359-a202-4e88-b23e-5be3fd002eee-openshift-service-ca\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:45.917071 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.917043    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3750d26b-7e8b-4220-bad9-5f560a339452-openshift-service-ca\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:45.917138 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.917090    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4977e359-a202-4e88-b23e-5be3fd002eee-metrics-certs\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:45.917183 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.917169    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4977e359-a202-4e88-b23e-5be3fd002eee-maas-api-tls\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:45.917215 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.917202    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt6wc\" (UniqueName: \"kubernetes.io/projected/3750d26b-7e8b-4220-bad9-5f560a339452-kube-api-access-gt6wc\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:45.917258 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.917222    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsnhp\" (UniqueName: \"kubernetes.io/projected/4977e359-a202-4e88-b23e-5be3fd002eee-kube-api-access-fsnhp\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:45.917258 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.917237    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-maas-api-tls\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:45.922176 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.922151    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"]
Oct 30 16:11:45.928098 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:45.928074    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"]
Oct 30 16:11:46.017830 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.017794    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4977e359-a202-4e88-b23e-5be3fd002eee-openshift-service-ca\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:46.018047 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.017841    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3750d26b-7e8b-4220-bad9-5f560a339452-openshift-service-ca\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:46.018047 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.017871    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4977e359-a202-4e88-b23e-5be3fd002eee-metrics-certs\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:46.018047 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.017902    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4977e359-a202-4e88-b23e-5be3fd002eee-maas-api-tls\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:46.018047 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.017940    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt6wc\" (UniqueName: \"kubernetes.io/projected/3750d26b-7e8b-4220-bad9-5f560a339452-kube-api-access-gt6wc\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:46.018047 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.017970    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsnhp\" (UniqueName: \"kubernetes.io/projected/4977e359-a202-4e88-b23e-5be3fd002eee-kube-api-access-fsnhp\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:46.018047 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.017998    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-maas-api-tls\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:46.018449 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.018127    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-metrics-certs\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:46.018745 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.018716    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4977e359-a202-4e88-b23e-5be3fd002eee-openshift-service-ca\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:46.018745 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.018724    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3750d26b-7e8b-4220-bad9-5f560a339452-openshift-service-ca\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:46.021069 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.021034    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4977e359-a202-4e88-b23e-5be3fd002eee-metrics-certs\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:46.029566 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.029535    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-metrics-certs\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:46.029912 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.029888    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4977e359-a202-4e88-b23e-5be3fd002eee-maas-api-tls\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:46.030538 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.030519    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-maas-api-tls\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:46.040857 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.040829    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt6wc\" (UniqueName: \"kubernetes.io/projected/3750d26b-7e8b-4220-bad9-5f560a339452-kube-api-access-gt6wc\") pod \"maas-api-5c48877c75-w5bmv\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:46.041109 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.041086    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsnhp\" (UniqueName: \"kubernetes.io/projected/4977e359-a202-4e88-b23e-5be3fd002eee-kube-api-access-fsnhp\") pod \"maas-api-595cc648c-sjnwh\" (UID: \"4977e359-a202-4e88-b23e-5be3fd002eee\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:46.195858 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.195817    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:46.212470 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.212431    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:46.499851 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.499821    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"]
Oct 30 16:11:46.500520 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:11:46.500494    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3750d26b_7e8b_4220_bad9_5f560a339452.slice/crio-f2e74c8e4fe0241905c16d821aea69efb5a96071f03f0305ff103c6a6a62135a WatchSource:0}: Error finding container f2e74c8e4fe0241905c16d821aea69efb5a96071f03f0305ff103c6a6a62135a: Status 404 returned error can't find the container with id f2e74c8e4fe0241905c16d821aea69efb5a96071f03f0305ff103c6a6a62135a
Oct 30 16:11:46.502237 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.502216    4585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 16:11:46.536628 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:46.536601    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"]
Oct 30 16:11:46.538450 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:11:46.538416    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4977e359_a202_4e88_b23e_5be3fd002eee.slice/crio-4f2fd384e500e494b4b093c215cce715d641351e3064c9c64e0432189e708704 WatchSource:0}: Error finding container 4f2fd384e500e494b4b093c215cce715d641351e3064c9c64e0432189e708704: Status 404 returned error can't find the container with id 4f2fd384e500e494b4b093c215cce715d641351e3064c9c64e0432189e708704
Oct 30 16:11:47.032389 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:47.027847    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh" event={"ID":"4977e359-a202-4e88-b23e-5be3fd002eee","Type":"ContainerStarted","Data":"3a704ba1e154fb7206f6987f8d88edda5aceb01ddf33058ee8a2f3715e1e759a"}
Oct 30 16:11:47.032389 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:47.027915    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh" event={"ID":"4977e359-a202-4e88-b23e-5be3fd002eee","Type":"ContainerStarted","Data":"4f2fd384e500e494b4b093c215cce715d641351e3064c9c64e0432189e708704"}
Oct 30 16:11:47.032389 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:47.028203    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:47.032389 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:47.029813    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv" event={"ID":"3750d26b-7e8b-4220-bad9-5f560a339452","Type":"ContainerStarted","Data":"c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d"}
Oct 30 16:11:47.032389 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:47.029865    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv" event={"ID":"3750d26b-7e8b-4220-bad9-5f560a339452","Type":"ContainerStarted","Data":"f2e74c8e4fe0241905c16d821aea69efb5a96071f03f0305ff103c6a6a62135a"}
Oct 30 16:11:47.032389 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:47.030322    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:47.080031 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:47.079976    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh" podStartSLOduration=1.682031404 podStartE2EDuration="2.079961404s" totalImagesPullingTime="397.93ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:11:47.078610519 +0000 UTC m=+2016.514214866" watchObservedRunningTime="2026-10-30 16:11:47.079961404 +0000 UTC m=+2016.515565787"
Oct 30 16:11:47.126097 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:47.126032    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv" podStartSLOduration=1.694138819 podStartE2EDuration="2.126016331s" totalImagesPullingTime="431.877512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 16:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:11:47.1242842 +0000 UTC m=+2016.559888548" watchObservedRunningTime="2026-10-30 16:11:47.126016331 +0000 UTC m=+2016.561620679"
Oct 30 16:11:47.960816 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:11:47.960782    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:11:53.040212 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.040185    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-sjnwh"
Oct 30 16:11:53.177216 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.177185    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"]
Oct 30 16:11:53.177410 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.177388    4585 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv" podUID="3750d26b-7e8b-4220-bad9-5f560a339452" containerName="maas-api" containerID="cri-o://c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d" gracePeriod=30
Oct 30 16:11:53.182735 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.182711    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:53.436826 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.436798    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:53.491993 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.491958    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3750d26b-7e8b-4220-bad9-5f560a339452-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3750d26b-7e8b-4220-bad9-5f560a339452-openshift-service-ca\") pod \"3750d26b-7e8b-4220-bad9-5f560a339452\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") "
Oct 30 16:11:53.492190 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.492143    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3750d26b-7e8b-4220-bad9-5f560a339452-kube-api-access-gt6wc\" (UniqueName: \"kubernetes.io/projected/3750d26b-7e8b-4220-bad9-5f560a339452-kube-api-access-gt6wc\") pod \"3750d26b-7e8b-4220-bad9-5f560a339452\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") "
Oct 30 16:11:53.492190 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.492182    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-metrics-certs\") pod \"3750d26b-7e8b-4220-bad9-5f560a339452\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") "
Oct 30 16:11:53.492319 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.492228    4585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-maas-api-tls\") pod \"3750d26b-7e8b-4220-bad9-5f560a339452\" (UID: \"3750d26b-7e8b-4220-bad9-5f560a339452\") "
Oct 30 16:11:53.492414 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.492384    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3750d26b-7e8b-4220-bad9-5f560a339452-openshift-service-ca" pod "3750d26b-7e8b-4220-bad9-5f560a339452" (UID: "3750d26b-7e8b-4220-bad9-5f560a339452"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 16:11:53.492607 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.492579    4585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3750d26b-7e8b-4220-bad9-5f560a339452-openshift-service-ca\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:53.494420 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.494390    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3750d26b-7e8b-4220-bad9-5f560a339452-kube-api-access-gt6wc" pod "3750d26b-7e8b-4220-bad9-5f560a339452" (UID: "3750d26b-7e8b-4220-bad9-5f560a339452"). InnerVolumeSpecName "kube-api-access-gt6wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 16:11:53.494515 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.494453    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-metrics-certs" pod "3750d26b-7e8b-4220-bad9-5f560a339452" (UID: "3750d26b-7e8b-4220-bad9-5f560a339452"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:11:53.494754 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.494734    4585 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-maas-api-tls" pod "3750d26b-7e8b-4220-bad9-5f560a339452" (UID: "3750d26b-7e8b-4220-bad9-5f560a339452"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 16:11:53.594073 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.593983    4585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-metrics-certs\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:53.594073 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.594021    4585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3750d26b-7e8b-4220-bad9-5f560a339452-maas-api-tls\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:53.594073 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:53.594030    4585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt6wc\" (UniqueName: \"kubernetes.io/projected/3750d26b-7e8b-4220-bad9-5f560a339452-kube-api-access-gt6wc\") on node \"ip-10-0-140-110.ec2.internal\" DevicePath \"\""
Oct 30 16:11:54.065081 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:54.064998    4585 generic.go:356] "Generic (PLEG): container finished" podID="3750d26b-7e8b-4220-bad9-5f560a339452" containerID="c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d" exitCode=0
Oct 30 16:11:54.065081 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:54.065049    4585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"
Oct 30 16:11:54.065581 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:54.065090    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv" event={"ID":"3750d26b-7e8b-4220-bad9-5f560a339452","Type":"ContainerDied","Data":"c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d"}
Oct 30 16:11:54.065581 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:54.065138    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv" event={"ID":"3750d26b-7e8b-4220-bad9-5f560a339452","Type":"ContainerDied","Data":"f2e74c8e4fe0241905c16d821aea69efb5a96071f03f0305ff103c6a6a62135a"}
Oct 30 16:11:54.065581 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:54.065158    4585 scope.go:122] "RemoveContainer" containerID="c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d"
Oct 30 16:11:54.074562 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:54.074537    4585 scope.go:122] "RemoveContainer" containerID="c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d"
Oct 30 16:11:54.074846 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:11:54.074825    4585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d\": container with ID starting with c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d not found: ID does not exist" containerID="c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d"
Oct 30 16:11:54.074919 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:54.074855    4585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d"} err="failed to get container status \"c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d\": rpc error: code = NotFound desc = could not find container \"c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d\": container with ID starting with c2f57ecc8297be72ce32f38a357ddd738b8edbc43d72d7964c054d98283c080d not found: ID does not exist"
Oct 30 16:11:54.117337 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:54.117307    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"]
Oct 30 16:11:54.129457 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:54.129424    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-w5bmv"]
Oct 30 16:11:54.700601 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:11:54.700566    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3750d26b-7e8b-4220-bad9-5f560a339452" path="/var/lib/kubelet/pods/3750d26b-7e8b-4220-bad9-5f560a339452/volumes"
Oct 30 16:12:17.962380 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:12:17.962327    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:12:44.536274 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:12:44.536232    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65cff8b9d5-r6lh9_f874c784-9b43-41c6-bdaf-b029d9ca989d/authorino/0.log"
Oct 30 16:12:47.959591 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:12:47.959555    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:12:56.445777 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:12:56.445748    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-2w4j6_0b99e03c-dfa7-49c7-8899-e8751735da37/server/0.log"
Oct 30 16:12:57.355202 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:12:57.355175    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd_252b401b-dddf-424b-9ff0-6c4bf5b31748/copy-manifests/0.log"
Oct 30 16:12:57.389896 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:12:57.389870    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd_252b401b-dddf-424b-9ff0-6c4bf5b31748/manager/0.log"
Oct 30 16:12:58.881214 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:12:58.881164    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-sjnwh_4977e359-a202-4e88-b23e-5be3fd002eee/maas-api/0.log"
Oct 30 16:12:59.269409 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:12:59.269381    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0567114c-b4jw4_cfb55879-1ae9-4f3a-930a-9d970559fa28/revoke-keys/0.log"
Oct 30 16:12:59.625392 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:12:59.625299    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3e31a506-hzpmw_6a7099d5-e741-4216-8774-e5dac6d82f81/revoke-keys/0.log"
Oct 30 16:12:59.810201 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:12:59.810175    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-452b6d7b-67d7r_519752e8-6e80-476c-81b9-3811bd540663/revoke-keys/0.log"
Oct 30 16:13:00.179013 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:00.178979    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-d2bc93bc-dnlk2_61ca2cb3-f6e9-402e-aba6-48a14871d95c/revoke-keys/0.log"
Oct 30 16:13:02.290005 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:02.289965    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65cff8b9d5-r6lh9_f874c784-9b43-41c6-bdaf-b029d9ca989d/authorino/0.log"
Oct 30 16:13:03.176246 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:03.176213    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-bsgcr_1d5cd07c-a347-4580-b819-d7677753c15c/manager/0.log"
Oct 30 16:13:04.830038 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:04.830012    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f6967bb6d-622sf_51a8df42-6e87-4caf-a776-0c69b707b18a/kube-auth-proxy/0.log"
Oct 30 16:13:05.637090 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:05.637059    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-fh66k_f6fa0c9c-0a86-4a03-8af0-561d339e6831/istio-proxy/0.log"
Oct 30 16:13:06.253527 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:06.253491    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-l7x5w_4adedae1-f191-4075-8059-805e373e4418/istio-proxy/0.log"
Oct 30 16:13:06.692848 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:06.692818    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-rtvxh_619e7d6d-2940-4118-81a9-13a16430f65b/istio-proxy/0.log"
Oct 30 16:13:08.481883 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:08.481852    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6_a63940b1-9bb4-4a37-bd76-c560b72869ee/main/0.log"
Oct 30 16:13:08.679545 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:08.679507    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-hwb9z_f1d46283-c4f5-4114-a114-44900ef4eb18/main/0.log"
Oct 30 16:13:09.080441 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:09.080407    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-hxckg_f2bc9d3b-4eb9-446f-98c2-1527620c7c05/main/0.log"
Oct 30 16:13:09.489615 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:09.489575    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf_b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d/main/0.log"
Oct 30 16:13:09.686165 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:09.686143    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq_84e32e1d-bae5-4233-ac38-50ff6ccaa774/main/0.log"
Oct 30 16:13:10.578489 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:10.578451    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65cff8b9d5-r6lh9_f874c784-9b43-41c6-bdaf-b029d9ca989d/authorino/0.log"
Oct 30 16:13:11.587650 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:11.587620    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 16:13:11.594568 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:11.594544    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sg7k4_89518dc7-2a05-4db9-b4f4-0662fd34d01b/ovn-acl-logging/0.log"
Oct 30 16:13:12.114735 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:12.114705    4585 scope.go:122] "RemoveContainer" containerID="4a6978658894596c06e7e1438bb7fc41a0a565675caea0d5d01bbd445a159cf1"
Oct 30 16:13:17.965295 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:13:17.965243    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:13:20.960265 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:20.960238    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-2w4j6_0b99e03c-dfa7-49c7-8899-e8751735da37/server/0.log"
Oct 30 16:13:21.851238 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:21.851207    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd_252b401b-dddf-424b-9ff0-6c4bf5b31748/copy-manifests/0.log"
Oct 30 16:13:21.886181 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:21.886149    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b7c6fc8bc-k4vmd_252b401b-dddf-424b-9ff0-6c4bf5b31748/manager/0.log"
Oct 30 16:13:23.141168 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:23.141120    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-sjnwh_4977e359-a202-4e88-b23e-5be3fd002eee/maas-api/0.log"
Oct 30 16:13:23.548366 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:23.548327    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0567114c-b4jw4_cfb55879-1ae9-4f3a-930a-9d970559fa28/revoke-keys/0.log"
Oct 30 16:13:23.932963 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:23.932895    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3e31a506-hzpmw_6a7099d5-e741-4216-8774-e5dac6d82f81/revoke-keys/0.log"
Oct 30 16:13:24.119989 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:24.119958    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-452b6d7b-67d7r_519752e8-6e80-476c-81b9-3811bd540663/revoke-keys/0.log"
Oct 30 16:13:24.489629 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:24.489596    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-d2bc93bc-dnlk2_61ca2cb3-f6e9-402e-aba6-48a14871d95c/revoke-keys/0.log"
Oct 30 16:13:26.194338 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:26.194308    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65cff8b9d5-r6lh9_f874c784-9b43-41c6-bdaf-b029d9ca989d/authorino/0.log"
Oct 30 16:13:27.050260 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:27.050215    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-bsgcr_1d5cd07c-a347-4580-b819-d7677753c15c/manager/0.log"
Oct 30 16:13:28.503890 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:28.503856    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f6967bb6d-622sf_51a8df42-6e87-4caf-a776-0c69b707b18a/kube-auth-proxy/0.log"
Oct 30 16:13:29.294197 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:29.294163    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-fh66k_f6fa0c9c-0a86-4a03-8af0-561d339e6831/istio-proxy/0.log"
Oct 30 16:13:29.918297 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:29.918269    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-l7x5w_4adedae1-f191-4075-8059-805e373e4418/istio-proxy/0.log"
Oct 30 16:13:30.327452 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:30.327374    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-rtvxh_619e7d6d-2940-4118-81a9-13a16430f65b/istio-proxy/0.log"
Oct 30 16:13:31.846297 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:31.846268    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-4lbg6_a63940b1-9bb4-4a37-bd76-c560b72869ee/main/0.log"
Oct 30 16:13:32.047999 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:32.047972    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-hwb9z_f1d46283-c4f5-4114-a114-44900ef4eb18/main/0.log"
Oct 30 16:13:32.425669 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:32.425640    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-hxckg_f2bc9d3b-4eb9-446f-98c2-1527620c7c05/main/0.log"
Oct 30 16:13:32.801694 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:32.801587    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-4fmnf_b6a5ffbd-0cf9-41a0-b596-4a2d131f6b7d/main/0.log"
Oct 30 16:13:32.993124 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:32.993090    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-tbmcq_84e32e1d-bae5-4233-ac38-50ff6ccaa774/main/0.log"
Oct 30 16:13:47.960779 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:13:47.960741    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:13:50.031506 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:50.031468    4585 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"]
Oct 30 16:13:50.043324 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:50.043297    4585 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-dnlk2"]
Oct 30 16:13:50.699844 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:13:50.699802    4585 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61ca2cb3-f6e9-402e-aba6-48a14871d95c" path="/var/lib/kubelet/pods/61ca2cb3-f6e9-402e-aba6-48a14871d95c/volumes"
Oct 30 16:14:17.961730 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:14:17.961685    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:14:47.959842 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:14:47.959797    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:15:01.684520 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:01.684446    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9l9gb_75385577-f877-463f-a0fc-c9e2b844e840/global-pull-secret-syncer/0.log"
Oct 30 16:15:02.771889 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:02.771851    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-trlcx_587ce512-7067-42a2-9cf1-2babd4feae72/konnectivity-agent/0.log"
Oct 30 16:15:03.171734 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:03.171699    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-110.ec2.internal_50080b7deadb8e30dee000b7f0bbadfa/haproxy/0.log"
Oct 30 16:15:08.954123 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:08.954087    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65cff8b9d5-r6lh9_f874c784-9b43-41c6-bdaf-b029d9ca989d/authorino/0.log"
Oct 30 16:15:09.507963 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:09.507932    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-bsgcr_1d5cd07c-a347-4580-b819-d7677753c15c/manager/0.log"
Oct 30 16:15:11.073153 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.073120    4585 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"]
Oct 30 16:15:11.073573 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.073492    4585 cpu_manager.go:414] "containerMap: removing container" podUID="3750d26b-7e8b-4220-bad9-5f560a339452" containerName="maas-api"
Oct 30 16:15:11.073573 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.073504    4585 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3750d26b-7e8b-4220-bad9-5f560a339452" containerName="maas-api"
Oct 30 16:15:11.073573 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.073570    4585 memory_manager.go:365] "RemoveStaleState removing state" podUID="3750d26b-7e8b-4220-bad9-5f560a339452" containerName="maas-api"
Oct 30 16:15:11.076562 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.076547    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.099883 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.099855    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"]
Oct 30 16:15:11.142516 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.142482    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btgxc\" (UniqueName: \"kubernetes.io/projected/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-kube-api-access-btgxc\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.142678 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.142530    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-podres\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.142678 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.142617    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-proc\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.142759 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.142707    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-sys\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.142794 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.142757    4585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-lib-modules\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.184031 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.184002    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vkvmm\"/\"openshift-service-ca.crt\""
Oct 30 16:15:11.184223 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.184209    4585 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vkvmm\"/\"default-dockercfg-w6wsh\""
Oct 30 16:15:11.184282 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.184263    4585 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vkvmm\"/\"kube-root-ca.crt\""
Oct 30 16:15:11.243242 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.243202    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-sys\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.243429 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.243265    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-lib-modules\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.243429 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.243321    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btgxc\" (UniqueName: \"kubernetes.io/projected/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-kube-api-access-btgxc\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.243429 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.243327    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-sys\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.243429 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.243396    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-podres\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.243578 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.243423    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-lib-modules\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.243578 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.243482    4585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-proc\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.243578 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.243492    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-podres\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.243578 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.243571    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-proc\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.269135 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.269107    4585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btgxc\" (UniqueName: \"kubernetes.io/projected/f9d7fc2c-7b7b-4322-ba31-9294381cbd0b-kube-api-access-btgxc\") pod \"perf-node-gather-daemonset-h4s47\" (UID: \"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b\") " pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.388966 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.388877    4585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:11.686487 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:11.686458    4585 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"]
Oct 30 16:15:11.687420 ip-10-0-140-110 kubenswrapper[4585]: W1030 16:15:11.687394    4585 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf9d7fc2c_7b7b_4322_ba31_9294381cbd0b.slice/crio-336dfc3542f652953439fa5546551ded2c20d9c05b5766faa47bcbf7a552d42e WatchSource:0}: Error finding container 336dfc3542f652953439fa5546551ded2c20d9c05b5766faa47bcbf7a552d42e: Status 404 returned error can't find the container with id 336dfc3542f652953439fa5546551ded2c20d9c05b5766faa47bcbf7a552d42e
Oct 30 16:15:12.054471 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:12.054386    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47" event={"ID":"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b","Type":"ContainerStarted","Data":"94237e5c81b6b762693243618b1e9232fbdd33f0ba1241586b54cc2e06d2d06c"}
Oct 30 16:15:12.054471 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:12.054440    4585 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47" event={"ID":"f9d7fc2c-7b7b-4322-ba31-9294381cbd0b","Type":"ContainerStarted","Data":"336dfc3542f652953439fa5546551ded2c20d9c05b5766faa47bcbf7a552d42e"}
Oct 30 16:15:12.054471 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:12.054463    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:12.096118 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:12.096068    4585 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47" podStartSLOduration=1.096055363 podStartE2EDuration="1.096055363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 16:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 16:15:12.094751901 +0000 UTC m=+2221.530356253" watchObservedRunningTime="2026-10-30 16:15:12.096055363 +0000 UTC m=+2221.531659711"
Oct 30 16:15:12.139242 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:12.139213    4585 scope.go:122] "RemoveContainer" containerID="819cb7466336ff5303fd2f3e54d440c8d3477a3da94f793a8b962e7b248978d8"
Oct 30 16:15:12.815234 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:12.815197    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-56849f5b7c-kj7wx_e88608ac-eae3-4fef-8cc0-8133c087bf38/metrics-server/0.log"
Oct 30 16:15:12.942323 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:12.942290    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-pdlnq_fee57912-cec9-4ffe-8cf6-5a96ae0a2469/monitoring-plugin/0.log"
Oct 30 16:15:13.459167 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:13.459125    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4tlkp_b6618867-c4d0-40c1-9b00-25f1cf5475c4/node-exporter/0.log"
Oct 30 16:15:13.581382 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:13.581331    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4tlkp_b6618867-c4d0-40c1-9b00-25f1cf5475c4/kube-rbac-proxy/0.log"
Oct 30 16:15:13.701582 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:13.701554    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4tlkp_b6618867-c4d0-40c1-9b00-25f1cf5475c4/init-textfile/0.log"
Oct 30 16:15:14.729242 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:14.729210    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54566ffd-8215-4bfa-80fb-3eabf5c68ee0/prometheus/0.log"
Oct 30 16:15:14.848088 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:14.848056    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54566ffd-8215-4bfa-80fb-3eabf5c68ee0/config-reloader/0.log"
Oct 30 16:15:14.969393 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:14.969367    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54566ffd-8215-4bfa-80fb-3eabf5c68ee0/thanos-sidecar/0.log"
Oct 30 16:15:15.102072 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:15.102002    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54566ffd-8215-4bfa-80fb-3eabf5c68ee0/kube-rbac-proxy-web/0.log"
Oct 30 16:15:15.230296 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:15.230265    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54566ffd-8215-4bfa-80fb-3eabf5c68ee0/kube-rbac-proxy/0.log"
Oct 30 16:15:15.355684 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:15.355614    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54566ffd-8215-4bfa-80fb-3eabf5c68ee0/kube-rbac-proxy-thanos/0.log"
Oct 30 16:15:15.481533 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:15.481498    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_54566ffd-8215-4bfa-80fb-3eabf5c68ee0/init-config-reloader/0.log"
Oct 30 16:15:15.608219 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:15.608121    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-4rvjk_f80c9a38-1db0-4a00-bd7f-ac7298f2119c/prometheus-operator/0.log"
Oct 30 16:15:15.735300 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:15.735276    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-4rvjk_f80c9a38-1db0-4a00-bd7f-ac7298f2119c/kube-rbac-proxy/0.log"
Oct 30 16:15:17.960246 ip-10-0-140-110 kubenswrapper[4585]: E1030 16:15:17.960211    4585 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-zknrn" containerName="extractor"
Oct 30 16:15:18.068553 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:18.068525    4585 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vkvmm/perf-node-gather-daemonset-h4s47"
Oct 30 16:15:25.355292 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:25.355259    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f6967bb6d-622sf_51a8df42-6e87-4caf-a776-0c69b707b18a/kube-auth-proxy/0.log"
Oct 30 16:15:25.635274 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:25.635197    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-fh66k_f6fa0c9c-0a86-4a03-8af0-561d339e6831/istio-proxy/0.log"
Oct 30 16:15:25.871099 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:25.871074    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-l7x5w_4adedae1-f191-4075-8059-805e373e4418/istio-proxy/0.log"
Oct 30 16:15:25.933243 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:25.933213    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bxct5_6a9a6ff0-5946-4635-9f97-c7be77c197c5/dns/0.log"
Oct 30 16:15:26.019196 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:26.019170    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-9dccdc9b7-rtvxh_619e7d6d-2940-4118-81a9-13a16430f65b/istio-proxy/0.log"
Oct 30 16:15:26.067191 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:26.067164    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bxct5_6a9a6ff0-5946-4635-9f97-c7be77c197c5/kube-rbac-proxy/0.log"
Oct 30 16:15:26.765763 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:26.765735    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngvd2_4cc7d376-7849-4d50-8c0f-3128e474b54c/dns-node-resolver/0.log"
Oct 30 16:15:28.135598 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:28.135564    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7d489c8c55-5ptpf_7a4454c5-1cc0-40d9-8e74-6adaf667fa66/registry/0.log"
Oct 30 16:15:28.584260 ip-10-0-140-110 kubenswrapper[4585]: I1030 16:15:28.584233    4585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xrrf9_630d1cfd-5d0a-41ec-bc4d-0d3d6b58b9f8/node-ca/0.log"