Nov 02 22:02:27.390207 ip-10-0-130-41 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 02 22:02:27.390216 ip-10-0-130-41 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 02 22:02:27.390223 ip-10-0-130-41 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 02 22:02:27.390456 ip-10-0-130-41 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 02 22:02:37.639485 ip-10-0-130-41 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 02 22:02:37.639506 ip-10-0-130-41 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d2c37894583c4ffb853b173f88d954db --
Nov 02 22:04:45.329692 ip-10-0-130-41 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 22:04:45.787968 ip-10-0-130-41 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 22:04:45.787968 ip-10-0-130-41 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 22:04:45.787968 ip-10-0-130-41 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 22:04:45.787968 ip-10-0-130-41 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 22:04:45.792227 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792144    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:04:45.792227 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792228    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792233    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792236    2583 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792239    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792243    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792245    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792248    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792251    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792254    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792256    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792259    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792262    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792264    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792267    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792270    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792273    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792282    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792285    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792287    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792290    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:04:45.792294 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792293    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792295    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792299    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792308    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792311    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792313    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792316    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792318    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792321    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792324    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792327    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792329    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792332    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792335    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792337    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792340    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792343    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792345    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792348    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792350    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:04:45.792806 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792353    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792355    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792358    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792360    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792375    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792378    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792381    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792383    2583 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792386    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792389    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792392    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792395    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792399    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792402    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792405    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792407    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792410    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792413    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792415    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792418    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:04:45.793274 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792421    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792423    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792426    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792429    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792432    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792435    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792437    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792440    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792443    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792445    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792448    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792450    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792454    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792458    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792462    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792464    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792467    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792470    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792472    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:04:45.793792 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792475    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792478    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792481    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792484    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792487    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792490    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792493    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792496    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792498    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792501    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792504    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792506    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792509    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792511    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792514    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792517    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792519    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792522    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792525    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:04:45.794247 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792528    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:04:45.794714 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792530    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:04:45.794714 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792533    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:04:45.794714 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792535    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:04:45.794714 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792538    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:04:45.794714 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792541    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:04:45.794714 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.792545    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:04:45.795292 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795278    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:04:45.795292 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795292    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795296    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795299    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795301    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795304    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795307    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795309    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795312    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795315    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795320    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795324    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795327    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795330    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795333    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795336    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795339    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795341    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795344    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795346    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:04:45.795343 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795349    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795353    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795355    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795358    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795361    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795376    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795379    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795382    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795384    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795387    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795390    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795392    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795395    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795397    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795400    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795404    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795408    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795411    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795414    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:04:45.795825 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795416    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795419    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795422    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795425    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795427    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795430    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795432    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795435    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795438    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795440    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795443    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795445    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795449    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795451    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795454    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795457    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795460    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795463    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795466    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795469    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:04:45.796281 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795471    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795474    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795477    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795479    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795481    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795484    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795487    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795489    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795492    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795495    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795497    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795500    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795502    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795505    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795507    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795512    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795515    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795517    2583 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795520    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795523    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:04:45.796796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795526    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795528    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795531    2583 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795534    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795537    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795539    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795542    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795544    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795547    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795550    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795552    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795555    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795558    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795561    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795564    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795566    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795569    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795572    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795574    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795577    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:04:45.797271 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795579    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795582    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795584    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795587    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795589    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795594    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.795597    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796100    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796105    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796109    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796111    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796114    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796116    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796119    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796122    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796126    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796128    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796131    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796134    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796136    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:04:45.797760 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796139    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796141    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796144    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796146    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796149    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796152    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796155    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796157    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796160    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796162    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796165    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796168    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796170    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796173    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796176    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796178    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796181    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796185    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796188    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796190    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:04:45.798238 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796193    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796196    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796199    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796201    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796204    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796206    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796209    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796212    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796214    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796217    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796220    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796222    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796225    2583 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796228    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796230    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796233    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796236    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796238    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796242    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:04:45.798749 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796245    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796249    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796252    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796255    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796257    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796260    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796262    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796265    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796267    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796270    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796274    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796277    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796279    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796282    2583 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796285    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796288    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796291    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796294    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796296    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796299    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:04:45.799199 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796302    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796305    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796307    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796310    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796312    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796315    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796317    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796320    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796322    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796324    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796327    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796329    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796332    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796335    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796337    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796340    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796342    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796345    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796347    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796350    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:04:45.799666 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796352    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796358    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796375    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796379    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796382    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796385    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796387    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796390    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796393    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796396    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796399    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796403    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796406    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.796409    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796509    2583 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796523    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796529    2583 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796534    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796538    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796542    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 22:04:45.800137 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796547    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796551    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796554    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796557    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796561    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796565    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796568    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796571    2583 flags.go:64] FLAG: --cgroup-root=""
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796574    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796577    2583 flags.go:64] FLAG: --client-ca-file=""
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796580    2583 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796583    2583 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796587    2583 flags.go:64] FLAG: --cluster-domain=""
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796591    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796595    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796599    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796602    2583 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796606    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796609    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796612    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796616    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796619    2583 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796623    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796626    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 22:04:45.800620 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796629    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796632    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796636    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796639    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796642    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796645    2583 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796648    2583 flags.go:64] FLAG: --enable-server="true"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796651    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796655    2583 flags.go:64] FLAG: --event-burst="100"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796659    2583 flags.go:64] FLAG: --event-qps="50"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796662    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796665    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796668    2583 flags.go:64] FLAG: --eviction-hard=""
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796673    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796677    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796680    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796683    2583 flags.go:64] FLAG: --eviction-soft=""
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796686    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796689    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796692    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796695    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796700    2583 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796704    2583 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796707    2583 flags.go:64] FLAG: --feature-gates=""
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796712    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 22:04:45.801244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796715    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796718    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796722    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796725    2583 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796728    2583 flags.go:64] FLAG: --help="false"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796731    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796735    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796738    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796741    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796745    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796748    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796751    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796754    2583 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796757    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796760    2583 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796763    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796766    2583 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796769    2583 flags.go:64] FLAG: --kube-reserved=""
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796772    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796775    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796779    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796781    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796784    2583 flags.go:64] FLAG: --lock-file=""
Nov 02 22:04:45.801902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796787    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796791    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796794    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796800    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796803    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796807    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796811    2583 flags.go:64] FLAG: --logging-format="text"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796814    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796818    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796820    2583 flags.go:64] FLAG: --manifest-url=""
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796823    2583 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796828    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796831    2583 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796835    2583 flags.go:64] FLAG: --max-pods="110"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796838    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796842    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796845    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796848    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796851    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796854    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796857    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796865    2583 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796868    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796871    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796874    2583 flags.go:64] FLAG: --pod-cidr=""
Nov 02 22:04:45.802571 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796877    2583 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796881    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796884    2583 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796887    2583 flags.go:64] FLAG: --port="10250"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796890    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796893    2583 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-081e350f0ef2d6c35"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796896    2583 flags.go:64] FLAG: --qos-reserved=""
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796899    2583 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796902    2583 flags.go:64] FLAG: --register-node="true"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796905    2583 flags.go:64] FLAG: --register-with-taints=""
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796908    2583 flags.go:64] FLAG: --registry-burst="10"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796911    2583 flags.go:64] FLAG: --registry-qps="5"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796916    2583 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796919    2583 flags.go:64] FLAG: --reserved-memory=""
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796923    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796926    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796929    2583 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796932    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796935    2583 flags.go:64] FLAG: --runonce="false"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796938    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796941    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796944    2583 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796947    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796951    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796955    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796959    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 22:04:45.803155 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796962    2583 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796965    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796968    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796971    2583 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796975    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796978    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796981    2583 flags.go:64] FLAG: --system-cgroups=""
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796984    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796989    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796993    2583 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796995    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.796999    2583 flags.go:64] FLAG: --tls-min-version=""
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.797002    2583 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.797005    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.797008    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.797011    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.797014    2583 flags.go:64] FLAG: --v="2"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.797018    2583 flags.go:64] FLAG: --version="false"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.797024    2583 flags.go:64] FLAG: --vmodule=""
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.797028    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.797031    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797173    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797177    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797180    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:04:45.803796 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797183    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797185    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797188    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797191    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797195    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797198    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797202    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797205    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797207    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797211    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797213    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797216    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797218    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797221    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797224    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797226    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797229    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797231    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797234    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:04:45.804374 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797236    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797239    2583 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797242    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797244    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797246    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797249    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797252    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797256    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797258    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797261    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797263    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797266    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797268    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797271    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797274    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797277    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797280    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797282    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797285    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797288    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:04:45.804882 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797291    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797294    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797296    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797299    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797301    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797304    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797308    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797310    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797313    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797315    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797318    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797320    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797323    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797325    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797328    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797330    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797333    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797336    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797338    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797341    2583 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:04:45.805538 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797344    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797346    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797349    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797351    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797354    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797357    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797359    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797375    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797378    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797381    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797383    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797386    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797388    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797391    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797394    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797396    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797399    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797401    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797405    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:04:45.806359 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797408    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797411    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797413    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797416    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797418    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797421    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797424    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797426    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797429    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797432    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797434    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797437    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797439    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797442    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797444    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797447    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797449    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797452    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797456    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:04:45.806850 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797460    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797463    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797466    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797470    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797472    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797475    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.797501    2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797537    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797542    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797546    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797549    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797552    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797556    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797559    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797561    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797564    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:04:45.807295 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797566    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797569    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797571    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797574    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797578    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797582    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797585    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797588    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797591    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797594    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797596    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797599    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797601    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797604    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797606    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797609    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797611    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797614    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:04:45.807690 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797618    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797621    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797624    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797627    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797629    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797632    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797635    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797638    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797641    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797643    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797647    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797650    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797653    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797656    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797659    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797661    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797665    2583 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797668    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797670    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797673    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:04:45.808113 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797675    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797678    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797681    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797683    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797686    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797688    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797691    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797693    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797696    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797698    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797701    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797703    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797706    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797708    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797711    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797713    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797716    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797718    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797721    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797739    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:04:45.808599 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797744    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797747    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797751    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797754    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797757    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797759    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797762    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797765    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797767    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797772    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797774    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797777    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797780    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797782    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797785    2583 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797787    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797790    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797799    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797803    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797806    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:04:45.809064 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797810    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797813    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797816    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797819    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797821    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797824    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797827    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797829    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797832    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797834    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797837    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797839    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797842    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797845    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797849    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797852    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797854    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797857    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.797860    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:04:45.809606 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.807402    2583 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.807420    2583 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.808928    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.808935    2583 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.808999    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809005    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809008    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809011    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809014    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809019    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809023    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809026    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809029    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809032    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809035    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809038    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809041    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809044    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809047    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:04:45.810060 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809050    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809053    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809055    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809058    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809061    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809064    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809067    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809070    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809073    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809075    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809078    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809081    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809084    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809086    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809090    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809093    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809095    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809098    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809101    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809104    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:04:45.810537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809106    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809109    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809112    2583 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809115    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809117    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809120    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809123    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809125    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809129    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809133    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809137    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809140    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809143    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809146    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809149    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809152    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809154    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809157    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809159    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:04:45.811000 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809162    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809165    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809167    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809170    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809173    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809175    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809178    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809181    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809184    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809187    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809190    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809193    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809195    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809198    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809201    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809203    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809206    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809209    2583 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809211    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809214    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:04:45.811504 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809216    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809219    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809222    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809224    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809227    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809230    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809233    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809236    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809238    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809241    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809243    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809246    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809249    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809251    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809254    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809256    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809259    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809262    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809265    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809268    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:04:45.811978 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809270    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809273    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809276    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809279    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809281    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809284    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809287    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809289    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809292    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809294    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809297    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809299    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.809329    2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809482    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809488    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809491    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:04:45.812454 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809494    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809497    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809500    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809503    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809506    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809509    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809511    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809514    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809517    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809520    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809522    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809526    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809529    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809532    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809534    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809537    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809540    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809542    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809545    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:04:45.812832 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809548    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809550    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809553    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809556    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809558    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809561    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809563    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809566    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809568    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809571    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809574    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809578    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809581    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809584    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809586    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809589    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809592    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809595    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809597    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:04:45.813315 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809600    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809602    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809605    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809608    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809610    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809613    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809616    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809619    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809623    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809628    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809631    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809634    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809636    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809639    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809642    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809644    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809647    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809650    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809652    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:04:45.813872 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809655    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809657    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809660    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809662    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809665    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809667    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809670    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809673    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809675    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809678    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809680    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809683    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809686    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809688    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809691    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809693    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809696    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809699    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809701    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809704    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:04:45.814336 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809707    2583 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809710    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809712    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809715    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809717    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809720    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809722    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809725    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809727    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809730    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809732    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809735    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809737    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809740    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809743    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809746    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809748    2583 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809751    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809754    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809756    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:04:45.814821 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809759    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809761    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809765    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809768    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809771    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809774    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.809797    2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809829    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809833    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809835    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809838    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809841    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809844    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809847    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809849    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809852    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:04:45.815289 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809855    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809857    2583 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809860    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809862    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809865    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809868    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809870    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809874    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809876    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809879    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809881    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809884    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809887    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809889    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809892    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809894    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809897    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809901    2583 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809903    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809906    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:04:45.815685 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809909    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809912    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809915    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809917    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809920    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809923    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809925    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809928    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809930    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809933    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809935    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809938    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809940    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809943    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809945    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809948    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809950    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809952    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809955    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:04:45.816152 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809958    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809960    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809963    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809966    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809968    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809971    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809973    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809976    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809978    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809981    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809984    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809986    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809989    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809991    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809994    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.809997    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810000    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810002    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810005    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:04:45.816603 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810007    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810011    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810015    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810017    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810020    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810023    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810025    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810029    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810031    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810034    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810036    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810039    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810041    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810044    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810046    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810049    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810051    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810054    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810058    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:04:45.817042 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810064    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810068    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810071    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810074    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810077    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810081    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810084    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810087    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810089    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810092    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810095    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810098    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810101    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810104    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810106    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810109    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810111    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810114    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810117    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:45.810119    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:04:45.817502 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.810183    2583 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 22:04:45.817990 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.813153    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 02 22:04:45.817990 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.814098    2583 server.go:1014] "Starting client certificate rotation"
Nov 02 22:04:45.817990 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.814198    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 22:04:45.817990 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.814976    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 22:04:45.817990 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.816508    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 22:04:45.817990 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.817675    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 22:04:45.831236 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.831217    2583 log.go:25] "Validated CRI v1 runtime API"
Nov 02 22:04:45.834148 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.834132    2583 log.go:25] "Validated CRI v1 image API"
Nov 02 22:04:45.835102 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.835081    2583 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 22:04:45.838982 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.838956    2583 fs.go:135] Filesystem UUIDs: map[3348d717-1d93-48bb-9667-14d0774f3849:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e74e0645-0c03-47ea-8b8d-a9684fff4664:/dev/nvme0n1p3]
Nov 02 22:04:45.839056 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.838980    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 02 22:04:45.846087 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.844571    2583 manager.go:217] Machine: {Timestamp:2026-11-02 22:04:45.843180531 +0000 UTC m=+0.382625342 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097616 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2550167a686dc53f373d2f1190a696 SystemUUID:ec255016-7a68-6dc5-3f37-3d2f1190a696 BootID:d2c37894-583c-4ffb-853b-173f88d954db Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:fe:ce:96:93 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:fe:ce:96:93 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:9f:11:83:7a:76 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 22:04:45.846087 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.845584    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 22:04:45.846087 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.845765    2583 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 22:04:45.849112 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.849083    2583 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 22:04:45.849283 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.849119    2583 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-41.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 22:04:45.849358 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.849289    2583 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 22:04:45.849358 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.849302    2583 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 22:04:45.849358 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.849318    2583 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 22:04:45.849358 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.849334    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 22:04:45.850349 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.850337    2583 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 22:04:45.853854 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.853840    2583 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 22:04:45.854003 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.853992    2583 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 22:04:45.854889 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.854877    2583 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 22:04:45.854943 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.854896    2583 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 22:04:45.854943 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.854911    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 22:04:45.854943 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.854923    2583 kubelet.go:405] "Adding apiserver pod source"
Nov 02 22:04:45.854943 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.854935    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 22:04:45.855327 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.855313    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 22:04:45.858981 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.858962    2583 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 22:04:45.865205 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.865192    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 22:04:45.866104 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866092    2583 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 22:04:45.866149 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866123    2583 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 22:04:45.866279 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866268    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 22:04:45.866310 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866290    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 22:04:45.866310 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866300    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 22:04:45.866310 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866310    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 22:04:45.866424 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866318    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 22:04:45.866424 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866339    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 22:04:45.866980 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866970    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 22:04:45.866980 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866980    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 22:04:45.867040 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866987    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 22:04:45.867040 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.866993    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 22:04:45.867040 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.867002    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 22:04:45.867040 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.867011    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 22:04:45.867807 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.867797    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 22:04:45.867807 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.867806    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 22:04:45.871846 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.871827    2583 server.go:1263] "Started kubelet"
Nov 02 22:04:45.872111 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.872069    2583 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 22:04:45.872194 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.871990    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 22:04:45.872258 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.872247    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 22:04:45.872693 ip-10-0-130-41 systemd[1]: Started Kubernetes Kubelet.
Nov 02 22:04:45.875813 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:45.875799    2583 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 22:04:45.878962 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:45.878944    2583 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 02 22:04:46.008996 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.008961    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-41.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 02 22:04:46.010054 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:46.008966    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-41.us-west-2.compute.internal.18e4599dcc404169  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-41.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-41.us-west-2.compute.internal,},FirstTimestamp:2026-11-02 22:04:45.871800681 +0000 UTC m=+0.411245487,LastTimestamp:2026-11-02 22:04:45.871800681 +0000 UTC m=+0.411245487,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-41.us-west-2.compute.internal,}"
Nov 02 22:04:46.013933 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.013912    2583 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 22:04:46.020201 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.020177    2583 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 22:04:46.020201 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.020190    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 22:04:46.020388 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:46.020268    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-41.us-west-2.compute.internal\" not found"
Nov 02 22:04:46.020941 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.020915    2583 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 22:04:46.021052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.020954    2583 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 22:04:46.021104 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.021068    2583 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 22:04:46.021104 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.021074    2583 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 22:04:46.021194 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.021140    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 22:04:46.021359 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.021339    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 22:04:46.021359 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.021358    2583 factory.go:55] Registering systemd factory
Nov 02 22:04:46.021506 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.021379    2583 factory.go:223] Registration of the systemd container factory successfully
Nov 02 22:04:46.021653 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.021635    2583 factory.go:153] Registering CRI-O factory
Nov 02 22:04:46.021653 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.021655    2583 factory.go:223] Registration of the crio container factory successfully
Nov 02 22:04:46.021800 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.021671    2583 factory.go:103] Registering Raw factory
Nov 02 22:04:46.021800 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.021683    2583 manager.go:1199] Started watching for new ooms in manager
Nov 02 22:04:46.023304 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.023280    2583 manager.go:319] Starting recovery of all containers
Nov 02 22:04:46.034006 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.033837    2583 manager.go:324] Recovery completed
Nov 02 22:04:46.038018 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.037978    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 22:04:46.039792 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.039770    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 22:04:46.039891 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.039811    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 22:04:46.039891 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.039824    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 22:04:46.040494 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.040483    2583 cpu_manager.go:226] "Starting" policy="none"
Nov 02 22:04:46.040494 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.040492    2583 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 22:04:46.040611 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.040499    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 22:04:46.040611 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.040509    2583 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 22:04:46.041161 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.041144    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 22:04:46.041212 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.041176    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 22:04:46.041212 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.041187    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 22:04:46.041992 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.041978    2583 policy_none.go:50] "Start"
Nov 02 22:04:46.042037 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.041995    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 22:04:46.042037 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.042005    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 22:04:46.043192 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.043181    2583 policy_none.go:44] "Start"
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.085057    2583 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.089306    2583 manager.go:341] "Starting Device Plugin manager"
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:46.089852    2583 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.089870    2583 server.go:87] "Starting device plugin registration server"
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.090132    2583 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.090141    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.090229    2583 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.090306    2583 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.090313    2583 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:46.090979    2583 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 22:04:46.091056 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:46.091029    2583 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-41.us-west-2.compute.internal\" not found"
Nov 02 22:04:46.092889 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.091763    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 22:04:46.092929 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.092912    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 22:04:46.092961 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.092933    2583 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 22:04:46.092961 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.092957    2583 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 22:04:46.093045 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:46.092994    2583 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 02 22:04:46.105189 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.105171    2583 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 22:04:46.105290 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.105224    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wsnx"
Nov 02 22:04:46.160018 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.159991    2583 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 22:04:46.190414 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.190390    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 22:04:46.191379 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.191343    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 22:04:46.191487 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.191395    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 22:04:46.191487 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.191411    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 22:04:46.191487 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.191442    2583 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.193516 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.193498    2583 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"]
Nov 02 22:04:46.193566 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.193552    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 22:04:46.194587 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.194571    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 22:04:46.194677 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.194601    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 22:04:46.194677 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.194616    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 22:04:46.196139 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.196124    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 22:04:46.196277 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.196262    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.196328 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.196292    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 22:04:46.196864 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.196850    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 22:04:46.196929 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.196863    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 22:04:46.196929 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.196882    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 22:04:46.196929 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.196897    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 22:04:46.197018 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.196887    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 22:04:46.197052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.197027    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 22:04:46.198175 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.198162    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.198226 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.198191    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 22:04:46.198868 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.198853    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 22:04:46.198927 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.198885    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 22:04:46.198927 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.198900    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 22:04:46.227616 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:46.227591    2583 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-41.us-west-2.compute.internal\" not found" node="ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.230954 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:46.230939    2583 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-41.us-west-2.compute.internal\" not found" node="ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.251044 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.250975    2583 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 22:04:46.251044 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.250994    2583 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 22:04:46.252358 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.252342    2583 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 22:04:46.252918 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.252889    2583 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.252918 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:46.252913    2583 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-41.us-west-2.compute.internal\": node \"ip-10-0-130-41.us-west-2.compute.internal\" not found"
Nov 02 22:04:46.321235 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.321167    2583 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.322333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.322308    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.322409 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.322341    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.322409 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.322358    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/376e7bd75c236d364d4749b5f447d5f4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"376e7bd75c236d364d4749b5f447d5f4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.422554 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.422521    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.422554 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.422553    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.422738 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.422572    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/376e7bd75c236d364d4749b5f447d5f4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"376e7bd75c236d364d4749b5f447d5f4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.422738 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.422632    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.422738 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.422636    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.422738 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.422674    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/376e7bd75c236d364d4749b5f447d5f4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"376e7bd75c236d364d4749b5f447d5f4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.460009 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.459949    2583 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 22:04:46.460893 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.460878    2583 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.531890 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.531849    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.535039 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.535011    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:04:46.573251 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.573179    2583 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 22:04:46.814323 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.814281    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 02 22:04:46.815082 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.814464    2583 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 22:04:46.815082 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.814516    2583 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 22:04:46.815082 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:46.814465    2583 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://ac0a2f50c2ab94c12807d3b1c760afba-8cfc36a42c20be03.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-130-41.us-west-2.compute.internal.18e4599dd6442cce\": read tcp 10.0.130.41:35198->16.59.97.209:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-130-41.us-west-2.compute.internal.18e4599dd6442cce  default   4815 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-41.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-41.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-41.us-west-2.compute.internal,},FirstTimestamp:2026-11-02 22:04:46 +0000 UTC,LastTimestamp:2026-11-02 22:04:46.191416802 +0000 UTC m=+0.730861610,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-41.us-west-2.compute.internal,}"
Nov 02 22:04:46.856753 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:46.856680    2583 apiserver.go:52] "Watching apiserver"
Nov 02 22:04:47.020430 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.020393    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 02 22:04:47.109827 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.109728    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 21:59:46 +0000 UTC" deadline="2028-05-04 05:08:37.576598186 +0000 UTC"
Nov 02 22:04:47.109827 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.109772    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13159h3m50.466829343s"
Nov 02 22:04:47.136795 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.136771    2583 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 22:04:47.137188 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.137167    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-v76fm","openshift-multus/multus-dkbhl","openshift-network-diagnostics/network-check-target-7j964","openshift-network-operator/iptables-alerter-bpxkm","kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal","openshift-dns/node-resolver-t5mqk","openshift-multus/multus-additional-cni-plugins-xbz67","openshift-multus/network-metrics-daemon-69fj9","kube-system/konnectivity-agent-ww4zl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal","openshift-ovn-kubernetes/ovnkube-node-5jljx","kube-system/global-pull-secret-syncer-78snq","openshift-cluster-node-tuning-operator/tuned-f4n2z"]
Nov 02 22:04:47.139073 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.139052    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.140537 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.140515    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.141834 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.141811    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:47.141915 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.141881    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:04:47.143251 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.143237    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.144672 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.144656    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.146055 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.146042    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.147472 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.147455    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:47.147551 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.147533    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:04:47.149029 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.149011    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:04:47.150830 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.150798    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.153924 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.153903    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.155520 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.155506    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:47.156119 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.156084    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:04:47.158278 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.158256    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.193295 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.193268    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 22:04:47.193295 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.193268    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 22:04:47.193451 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.193273    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zcntw\""
Nov 02 22:04:47.194649 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.194628    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 22:04:47.194776 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.194690    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 22:04:47.194776 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.194717    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 22:04:47.194776 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.194751    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 22:04:47.194922 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.194912    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 22:04:47.194976 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.194917    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-klqgw\""
Nov 02 22:04:47.195008 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.194977    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 22:04:47.197054 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.197037    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 22:04:47.197103 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.197072    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 22:04:47.197103 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.197094    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4ts5p\""
Nov 02 22:04:47.197169 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.197102    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:04:47.198230 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.198214    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 22:04:47.198324 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.198311    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rksbh\""
Nov 02 22:04:47.222764 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.222744    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 22:04:47.228303 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228282    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovn-node-metrics-cert\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.228417 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228311    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-multus\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.228417 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228332    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-etc-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.228417 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-log-socket\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.228417 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228375    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-script-lib\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.228613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-dbus\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:47.228613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228500    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-run\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.228613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228517    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-node-log\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.228613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228535    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c94vr\" (UniqueName: \"kubernetes.io/projected/1e801619-9294-420b-8ad5-cd1005832f16-kube-api-access-c94vr\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.228613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-netns\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.228613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228568    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r82z\" (UniqueName: \"kubernetes.io/projected/0caf91c1-266c-405f-92b4-070981db24c3-kube-api-access-4r82z\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.228613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228586    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21f54242-5bab-4162-a435-85a080f8685c-hosts-file\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228636    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21f54242-5bab-4162-a435-85a080f8685c-tmp-dir\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228655    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-sys\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228671    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e801619-9294-420b-8ad5-cd1005832f16-iptables-alerter-script\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228688    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e801619-9294-420b-8ad5-cd1005832f16-host-slash\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228703    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4843105e-92f4-4b83-83fd-06e7a217e009-agent-certs\") pod \"konnectivity-agent-ww4zl\" (UID: \"4843105e-92f4-4b83-83fd-06e7a217e009\") " pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228717    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-lib-modules\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228733    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-netd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228748    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsh4k\" (UniqueName: \"kubernetes.io/projected/9cfdf849-3285-4c0c-bcc8-2401667a460b-kube-api-access-tsh4k\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228825    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4843105e-92f4-4b83-83fd-06e7a217e009-konnectivity-ca\") pod \"konnectivity-agent-ww4zl\" (UID: \"4843105e-92f4-4b83-83fd-06e7a217e009\") " pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228865    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6f7s\" (UniqueName: \"kubernetes.io/projected/6b703071-5807-41b5-afe0-a2c85e23af47-kube-api-access-d6f7s\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228896    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228924    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-multus-daemon-config\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.228945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228950    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-system-cni-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.228977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-systemd-units\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229005    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dqqc\" (UniqueName: \"kubernetes.io/projected/713aba87-e440-4d2f-a339-52b3f222fa97-kube-api-access-8dqqc\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229033    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtlnb\" (UniqueName: \"kubernetes.io/projected/56b1aa47-8106-416e-869d-bfc5c10ef795-kube-api-access-jtlnb\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229061    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229084    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-conf\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229108    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-device-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229127    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-socket-dir-parent\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229154    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-multus-certs\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229190    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysconfig\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229217    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-kubelet\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229252    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdx65\" (UniqueName: \"kubernetes.io/projected/21f54242-5bab-4162-a435-85a080f8685c-kube-api-access-jdx65\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229284    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-ovn\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229321    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-env-overrides\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229352    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229397    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-kubelet-config\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:47.229507 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229414    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-bin\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229430    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-cni-binary-copy\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229450    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cnibin\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229491    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229579    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-config\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229644    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-cnibin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229694    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-bin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229710    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nff6m\" (UniqueName: \"kubernetes.io/projected/fc25374f-cb3c-45ce-a523-af6fa11b570e-kube-api-access-nff6m\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229735    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-systemd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229757    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229774    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/713aba87-e440-4d2f-a339-52b3f222fa97-host\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229796    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229822    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-registration-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229872    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-tmp\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.230007 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229892    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-netns\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229913    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229936    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/713aba87-e440-4d2f-a339-52b3f222fa97-serviceca\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229953    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-socket-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229973    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-system-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.229992    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-k8s-cni-cncf-io\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-conf-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230021    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-etc-kubernetes\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230041    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-etc-selinux\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dq8x\" (UniqueName: \"kubernetes.io/projected/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-kube-api-access-5dq8x\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230133    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-kubernetes\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230147    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230169    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-host\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230190    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-var-lib-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:47.230589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230224    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-tuned\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.231052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230250    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.231052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-sys-fs\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.231052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230296    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-kubelet\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.231052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230311    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-hostroot\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.231052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230338    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-modprobe-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.231052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230357    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-systemd\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.231052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230401    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-var-lib-kubelet\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.231052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.230444    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-slash\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.244857 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.244837    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rsfpb\""
Nov 02 22:04:47.244953 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.244940    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 22:04:47.245014 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.244955    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 22:04:47.245974 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.245953    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fcg7g\""
Nov 02 22:04:47.245974 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.245957    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 22:04:47.246109 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246097    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 22:04:47.246154 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246136    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 22:04:47.246201 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246156    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 22:04:47.246266 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246248    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 22:04:47.246327 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246267    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4rxp8\""
Nov 02 22:04:47.246327 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246255    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 22:04:47.246403 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246332    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 22:04:47.246486 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246472    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-656w2\""
Nov 02 22:04:47.246540 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246484    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 22:04:47.246540 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246516    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:04:47.246540 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246533    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 22:04:47.246677 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246535    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 22:04:47.246677 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246558    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 22:04:47.246677 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246569    2583 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 22:04:47.246677 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246574    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 22:04:47.246677 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246625    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cvfcn"
Nov 02 22:04:47.246858 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.246739    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-rrhfb\""
Nov 02 22:04:47.331632 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331599    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.331632 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-multus-daemon-config\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.331875 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331657    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-system-cni-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.331875 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331682    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-systemd-units\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.331875 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331733    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.331875 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331749    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dqqc\" (UniqueName: \"kubernetes.io/projected/713aba87-e440-4d2f-a339-52b3f222fa97-kube-api-access-8dqqc\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.331875 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331773    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-system-cni-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.331875 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331834    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtlnb\" (UniqueName: \"kubernetes.io/projected/56b1aa47-8106-416e-869d-bfc5c10ef795-kube-api-access-jtlnb\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:47.331875 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:47.332215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-conf\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.332215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331949    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-systemd-units\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.332215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.331960    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-device-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.332215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332076    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-device-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.332215 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.332055    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 22:04:47.332215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332137    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-socket-dir-parent\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.332215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332205    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-multus-certs\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.332215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332210    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-conf\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332229    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-socket-dir-parent\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332239    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-multus-certs\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332238    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysconfig\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332280    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysconfig\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.332284    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret podName:e47f56ea-3a73-4ff0-965d-3a21319aef2e nodeName:}" failed. No retries permitted until 2026-11-02 22:04:47.832243933 +0000 UTC m=+2.371688744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret") pod "global-pull-secret-syncer-78snq" (UID: "e47f56ea-3a73-4ff0-965d-3a21319aef2e") : object "kube-system"/"original-pull-secret" not registered
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332337    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-kubelet\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdx65\" (UniqueName: \"kubernetes.io/projected/21f54242-5bab-4162-a435-85a080f8685c-kube-api-access-jdx65\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332422    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-kubelet\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-ovn\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332501    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-env-overrides\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332507    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-ovn\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.332565 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332474    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-multus-daemon-config\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332621    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332652    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-kubelet-config\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-bin\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332731    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-cni-binary-copy\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332754    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cnibin\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332781    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-bin\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332786    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-kubelet-config\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332831    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cnibin\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332781    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332848    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332875    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-config\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332892    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332897    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-cnibin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332940    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-cnibin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.332966    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-bin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.333031 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333009    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nff6m\" (UniqueName: \"kubernetes.io/projected/fc25374f-cb3c-45ce-a523-af6fa11b570e-kube-api-access-nff6m\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333052    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-env-overrides\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333074    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-bin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-systemd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333150    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-systemd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/713aba87-e440-4d2f-a339-52b3f222fa97-host\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333232    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333247    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-registration-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333292    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-cni-binary-copy\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333303    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-tmp\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.333359    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.333433    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs podName:56b1aa47-8106-416e-869d-bfc5c10ef795 nodeName:}" failed. No retries permitted until 2026-11-02 22:04:47.8334152 +0000 UTC m=+2.372859998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs") pod "network-metrics-daemon-69fj9" (UID: "56b1aa47-8106-416e-869d-bfc5c10ef795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333441    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-netns\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333465    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.333845 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333499    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-registration-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333487    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-netns\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333503    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-config\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333722    2583 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333827    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333836    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/713aba87-e440-4d2f-a339-52b3f222fa97-host\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333853    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/713aba87-e440-4d2f-a339-52b3f222fa97-serviceca\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-socket-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333888    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-system-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333909    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-k8s-cni-cncf-io\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333940    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-conf-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333960    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-system-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333851    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333986    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-conf-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333989    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-k8s-cni-cncf-io\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.333976    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334008    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-socket-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334029    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-etc-kubernetes\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.334627 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334061    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-etc-selinux\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334076    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-etc-kubernetes\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334087    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dq8x\" (UniqueName: \"kubernetes.io/projected/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-kube-api-access-5dq8x\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334147    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-kubernetes\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334172    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-etc-selinux\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334174    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334204    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-kubernetes\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334203    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-host\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334240    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/713aba87-e440-4d2f-a339-52b3f222fa97-serviceca\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334301    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-var-lib-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334325    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-var-lib-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334306    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334325    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-host\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334358    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334401    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-tuned\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334430    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.335466 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334454    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-sys-fs\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334482    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-kubelet\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334507    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-hostroot\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334530    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-sys-fs\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334532    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-modprobe-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334532    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-kubelet\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334562    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-hostroot\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334627    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-modprobe-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334633    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334682    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334694    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-systemd\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334725    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-systemd\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334748    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-var-lib-kubelet\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334796    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-var-lib-kubelet\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334809    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-slash\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334854    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovn-node-metrics-cert\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-multus\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334894    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-slash\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.336344 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334917    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-etc-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334938    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-log-socket\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-script-lib\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.334970    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-multus\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335009    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-dbus\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335028    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-etc-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335047    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-run\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335084    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-node-log\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335113    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-dbus\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c94vr\" (UniqueName: \"kubernetes.io/projected/1e801619-9294-420b-8ad5-cd1005832f16-kube-api-access-c94vr\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335033    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-log-socket\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335131    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-run\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-netns\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335174    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-node-log\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335252    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r82z\" (UniqueName: \"kubernetes.io/projected/0caf91c1-266c-405f-92b4-070981db24c3-kube-api-access-4r82z\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335293    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-netns\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335371    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21f54242-5bab-4162-a435-85a080f8685c-hosts-file\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335397    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21f54242-5bab-4162-a435-85a080f8685c-tmp-dir\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.337273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335456    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21f54242-5bab-4162-a435-85a080f8685c-hosts-file\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335458    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-sys\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335516    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e801619-9294-420b-8ad5-cd1005832f16-iptables-alerter-script\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335555    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e801619-9294-420b-8ad5-cd1005832f16-host-slash\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4843105e-92f4-4b83-83fd-06e7a217e009-agent-certs\") pod \"konnectivity-agent-ww4zl\" (UID: \"4843105e-92f4-4b83-83fd-06e7a217e009\") " pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335619    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-lib-modules\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-netd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsh4k\" (UniqueName: \"kubernetes.io/projected/9cfdf849-3285-4c0c-bcc8-2401667a460b-kube-api-access-tsh4k\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335703    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4843105e-92f4-4b83-83fd-06e7a217e009-konnectivity-ca\") pod \"konnectivity-agent-ww4zl\" (UID: \"4843105e-92f4-4b83-83fd-06e7a217e009\") " pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335746    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6f7s\" (UniqueName: \"kubernetes.io/projected/6b703071-5807-41b5-afe0-a2c85e23af47-kube-api-access-d6f7s\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335746    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-script-lib\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335809    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21f54242-5bab-4162-a435-85a080f8685c-tmp-dir\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335903    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-sys\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.335998    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-netd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.336026    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-lib-modules\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.336078    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e801619-9294-420b-8ad5-cd1005832f16-host-slash\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.337786    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4843105e-92f4-4b83-83fd-06e7a217e009-konnectivity-ca\") pod \"konnectivity-agent-ww4zl\" (UID: \"4843105e-92f4-4b83-83fd-06e7a217e009\") " pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.337824    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-tuned\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.338115 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.337872    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-tmp\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.338786 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.337956    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovn-node-metrics-cert\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.338786 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.338071    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e801619-9294-420b-8ad5-cd1005832f16-iptables-alerter-script\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.338786 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.338370    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4843105e-92f4-4b83-83fd-06e7a217e009-agent-certs\") pod \"konnectivity-agent-ww4zl\" (UID: \"4843105e-92f4-4b83-83fd-06e7a217e009\") " pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:04:47.394466 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.394361    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 22:04:47.394466 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.394421    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 22:04:47.394466 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.394438    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dmjvr for pod openshift-network-diagnostics/network-check-target-7j964: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:04:47.394733 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.394524    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr podName:3505867a-1caf-4cef-b570-841afd5f7d85 nodeName:}" failed. No retries permitted until 2026-11-02 22:04:47.894500534 +0000 UTC m=+2.433945356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dmjvr" (UniqueName: "kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr") pod "network-check-target-7j964" (UID: "3505867a-1caf-4cef-b570-841afd5f7d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:04:47.396056 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.396015    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dqqc\" (UniqueName: \"kubernetes.io/projected/713aba87-e440-4d2f-a339-52b3f222fa97-kube-api-access-8dqqc\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.396189 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.396115    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdx65\" (UniqueName: \"kubernetes.io/projected/21f54242-5bab-4162-a435-85a080f8685c-kube-api-access-jdx65\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.396258 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.396199    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtlnb\" (UniqueName: \"kubernetes.io/projected/56b1aa47-8106-416e-869d-bfc5c10ef795-kube-api-access-jtlnb\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:47.396707 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.396678    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nff6m\" (UniqueName: \"kubernetes.io/projected/fc25374f-cb3c-45ce-a523-af6fa11b570e-kube-api-access-nff6m\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.397107 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.397074    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dq8x\" (UniqueName: \"kubernetes.io/projected/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-kube-api-access-5dq8x\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.397965 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.397944    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r82z\" (UniqueName: \"kubernetes.io/projected/0caf91c1-266c-405f-92b4-070981db24c3-kube-api-access-4r82z\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.398481 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.398461    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6f7s\" (UniqueName: \"kubernetes.io/projected/6b703071-5807-41b5-afe0-a2c85e23af47-kube-api-access-d6f7s\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.398826 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.398804    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c94vr\" (UniqueName: \"kubernetes.io/projected/1e801619-9294-420b-8ad5-cd1005832f16-kube-api-access-c94vr\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.398887 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.398810    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsh4k\" (UniqueName: \"kubernetes.io/projected/9cfdf849-3285-4c0c-bcc8-2401667a460b-kube-api-access-tsh4k\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.468707 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.468665    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:04:47.485123 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.485097    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:04:47.505632 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.505597    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:04:47.530601 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.530555    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:04:47.536881 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.536859    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:04:47.552287 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.552259    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:04:47.579055 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.579028    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:04:47.598160 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.598139    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dkbhl"
Nov 02 22:04:47.603888 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.603867    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:04:47.699453 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.699364    2583 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 22:04:47.773878 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:47.773837    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657ace375fa97dc1e56f2b6e64a6ac61.slice/crio-7c4defbccef486e8f04eaf189fd4286e3d2f77ed2bedff1b3306e43abcc777f6 WatchSource:0}: Error finding container 7c4defbccef486e8f04eaf189fd4286e3d2f77ed2bedff1b3306e43abcc777f6: Status 404 returned error can't find the container with id 7c4defbccef486e8f04eaf189fd4286e3d2f77ed2bedff1b3306e43abcc777f6
Nov 02 22:04:47.774277 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:47.774254    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod376e7bd75c236d364d4749b5f447d5f4.slice/crio-8a042539c600ac0bfc6d488d141c75bfc471cfe6c9710d3aaad6a3a68c93b6ae WatchSource:0}: Error finding container 8a042539c600ac0bfc6d488d141c75bfc471cfe6c9710d3aaad6a3a68c93b6ae: Status 404 returned error can't find the container with id 8a042539c600ac0bfc6d488d141c75bfc471cfe6c9710d3aaad6a3a68c93b6ae
Nov 02 22:04:47.778498 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.778477    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:04:47.839612 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.839555    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:47.840294 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.839661    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:47.840294 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.839749    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:04:47.840294 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.839761    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 22:04:47.840294 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.839826    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret podName:e47f56ea-3a73-4ff0-965d-3a21319aef2e nodeName:}" failed. No retries permitted until 2026-11-02 22:04:48.839808924 +0000 UTC m=+3.379253722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret") pod "global-pull-secret-syncer-78snq" (UID: "e47f56ea-3a73-4ff0-965d-3a21319aef2e") : object "kube-system"/"original-pull-secret" not registered
Nov 02 22:04:47.840294 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.839841    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs podName:56b1aa47-8106-416e-869d-bfc5c10ef795 nodeName:}" failed. No retries permitted until 2026-11-02 22:04:48.839833725 +0000 UTC m=+3.379278521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs") pod "network-metrics-daemon-69fj9" (UID: "56b1aa47-8106-416e-869d-bfc5c10ef795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:04:47.940273 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:47.940235    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:47.940505 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.940434    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 22:04:47.940505 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.940455    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 22:04:47.940505 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.940466    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dmjvr for pod openshift-network-diagnostics/network-check-target-7j964: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:04:47.940618 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:47.940518    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr podName:3505867a-1caf-4cef-b570-841afd5f7d85 nodeName:}" failed. No retries permitted until 2026-11-02 22:04:48.940503774 +0000 UTC m=+3.479948574 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmjvr" (UniqueName: "kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr") pod "network-check-target-7j964" (UID: "3505867a-1caf-4cef-b570-841afd5f7d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:04:48.097358 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:48.097255    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal" event={"ID":"376e7bd75c236d364d4749b5f447d5f4","Type":"ContainerStarted","Data":"8a042539c600ac0bfc6d488d141c75bfc471cfe6c9710d3aaad6a3a68c93b6ae"}
Nov 02 22:04:48.098401 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:48.098355    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal" event={"ID":"657ace375fa97dc1e56f2b6e64a6ac61","Type":"ContainerStarted","Data":"7c4defbccef486e8f04eaf189fd4286e3d2f77ed2bedff1b3306e43abcc777f6"}
Nov 02 22:04:48.248730 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:48.248684    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 21:59:47 +0000 UTC" deadline="2028-06-30 19:36:30.57037716 +0000 UTC"
Nov 02 22:04:48.248730 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:48.248716    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14541h31m42.321664391s"
Nov 02 22:04:48.418927 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:48.418893    2583 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 22:04:48.419084 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:48.418959    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e801619_9294_420b_8ad5_cd1005832f16.slice/crio-2505217cb809530edf93577485a6d4bc642d28d6b4e78eaac5490e713a1d207c WatchSource:0}: Error finding container 2505217cb809530edf93577485a6d4bc642d28d6b4e78eaac5490e713a1d207c: Status 404 returned error can't find the container with id 2505217cb809530edf93577485a6d4bc642d28d6b4e78eaac5490e713a1d207c
Nov 02 22:04:48.443811 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:48.443777    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod713aba87_e440_4d2f_a339_52b3f222fa97.slice/crio-0f1d9b7623f79f537e0583c362f333dead9b34cd227f2a6e7abec4c7d99b780e WatchSource:0}: Error finding container 0f1d9b7623f79f537e0583c362f333dead9b34cd227f2a6e7abec4c7d99b780e: Status 404 returned error can't find the container with id 0f1d9b7623f79f537e0583c362f333dead9b34cd227f2a6e7abec4c7d99b780e
Nov 02 22:04:48.469549 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:48.469519    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2bc1cd9_8631_4e6e_82be_053f3d8df4d4.slice/crio-cef0782836670b2f197f2f7b3dc587919b667417fa7d5bd22f36b9d04e71f51a WatchSource:0}: Error finding container cef0782836670b2f197f2f7b3dc587919b667417fa7d5bd22f36b9d04e71f51a: Status 404 returned error can't find the container with id cef0782836670b2f197f2f7b3dc587919b667417fa7d5bd22f36b9d04e71f51a
Nov 02 22:04:48.548549 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:48.548514    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0caf91c1_266c_405f_92b4_070981db24c3.slice/crio-21885d4b8115c0caba9a6b2aa5f7ab82895b1ca3d69ee5d45f6767938031df80 WatchSource:0}: Error finding container 21885d4b8115c0caba9a6b2aa5f7ab82895b1ca3d69ee5d45f6767938031df80: Status 404 returned error can't find the container with id 21885d4b8115c0caba9a6b2aa5f7ab82895b1ca3d69ee5d45f6767938031df80
Nov 02 22:04:48.549449 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:48.549414    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b703071_5807_41b5_afe0_a2c85e23af47.slice/crio-ae736b0b267bb64d8f407bf2cc055a3f7ebba2fc492253160e5be23f63236a0e WatchSource:0}: Error finding container ae736b0b267bb64d8f407bf2cc055a3f7ebba2fc492253160e5be23f63236a0e: Status 404 returned error can't find the container with id ae736b0b267bb64d8f407bf2cc055a3f7ebba2fc492253160e5be23f63236a0e
Nov 02 22:04:48.642229 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:48.642198    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4843105e_92f4_4b83_83fd_06e7a217e009.slice/crio-52556c04a276b16fcc795dc1809d30cf1936abed59a4bb77d93375e921789a10 WatchSource:0}: Error finding container 52556c04a276b16fcc795dc1809d30cf1936abed59a4bb77d93375e921789a10: Status 404 returned error can't find the container with id 52556c04a276b16fcc795dc1809d30cf1936abed59a4bb77d93375e921789a10
Nov 02 22:04:48.649904 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:04:48.649877    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc25374f_cb3c_45ce_a523_af6fa11b570e.slice/crio-0d4af4316992d374168bcf49437dfd025bdd15cef82bc4ac6103726c9e9bd557 WatchSource:0}: Error finding container 0d4af4316992d374168bcf49437dfd025bdd15cef82bc4ac6103726c9e9bd557: Status 404 returned error can't find the container with id 0d4af4316992d374168bcf49437dfd025bdd15cef82bc4ac6103726c9e9bd557
Nov 02 22:04:48.846644 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:48.846603    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:48.847057 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:48.846711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:48.847057 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:48.846752    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:04:48.847057 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:48.846759    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 22:04:48.847057 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:48.846813    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs podName:56b1aa47-8106-416e-869d-bfc5c10ef795 nodeName:}" failed. No retries permitted until 2026-11-02 22:04:50.846797517 +0000 UTC m=+5.386242328 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs") pod "network-metrics-daemon-69fj9" (UID: "56b1aa47-8106-416e-869d-bfc5c10ef795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:04:48.847057 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:48.846832    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret podName:e47f56ea-3a73-4ff0-965d-3a21319aef2e nodeName:}" failed. No retries permitted until 2026-11-02 22:04:50.846823978 +0000 UTC m=+5.386268773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret") pod "global-pull-secret-syncer-78snq" (UID: "e47f56ea-3a73-4ff0-965d-3a21319aef2e") : object "kube-system"/"original-pull-secret" not registered
Nov 02 22:04:48.947052 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:48.946969    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:48.947209 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:48.947153    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 22:04:48.947209 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:48.947174    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 22:04:48.947209 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:48.947185    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dmjvr for pod openshift-network-diagnostics/network-check-target-7j964: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:04:48.947326 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:48.947239    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr podName:3505867a-1caf-4cef-b570-841afd5f7d85 nodeName:}" failed. No retries permitted until 2026-11-02 22:04:50.947224533 +0000 UTC m=+5.486669344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmjvr" (UniqueName: "kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr") pod "network-check-target-7j964" (UID: "3505867a-1caf-4cef-b570-841afd5f7d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:04:49.094268 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.094231    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:49.094555 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.094525    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:49.094670 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:49.094622    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:04:49.094751 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.094677    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:49.094829 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:49.094682    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:04:49.094829 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:49.094806    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:04:49.102531 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.102498    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t5mqk" event={"ID":"21f54242-5bab-4162-a435-85a080f8685c","Type":"ContainerStarted","Data":"6031f68c17e487665f7a6e0c5ac48c2ec5b776e6585078c10c33f67c143311d8"}
Nov 02 22:04:49.103929 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.103898    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ww4zl" event={"ID":"4843105e-92f4-4b83-83fd-06e7a217e009","Type":"ContainerStarted","Data":"52556c04a276b16fcc795dc1809d30cf1936abed59a4bb77d93375e921789a10"}
Nov 02 22:04:49.105566 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.105524    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbz67" event={"ID":"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4","Type":"ContainerStarted","Data":"cef0782836670b2f197f2f7b3dc587919b667417fa7d5bd22f36b9d04e71f51a"}
Nov 02 22:04:49.106973 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.106945    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx" event={"ID":"9cfdf849-3285-4c0c-bcc8-2401667a460b","Type":"ContainerStarted","Data":"8d2c2bf0dd57b393bea6095f331f626d58b06f89e8c1b15cb39afa7eefdcff4c"}
Nov 02 22:04:49.109128 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.109100    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bpxkm" event={"ID":"1e801619-9294-420b-8ad5-cd1005832f16","Type":"ContainerStarted","Data":"2505217cb809530edf93577485a6d4bc642d28d6b4e78eaac5490e713a1d207c"}
Nov 02 22:04:49.110676 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.110650    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f4n2z" event={"ID":"fc25374f-cb3c-45ce-a523-af6fa11b570e","Type":"ContainerStarted","Data":"0d4af4316992d374168bcf49437dfd025bdd15cef82bc4ac6103726c9e9bd557"}
Nov 02 22:04:49.111987 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.111960    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr" event={"ID":"6b703071-5807-41b5-afe0-a2c85e23af47","Type":"ContainerStarted","Data":"ae736b0b267bb64d8f407bf2cc055a3f7ebba2fc492253160e5be23f63236a0e"}
Nov 02 22:04:49.114053 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.114030    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dkbhl" event={"ID":"0caf91c1-266c-405f-92b4-070981db24c3","Type":"ContainerStarted","Data":"21885d4b8115c0caba9a6b2aa5f7ab82895b1ca3d69ee5d45f6767938031df80"}
Nov 02 22:04:49.115459 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.115433    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v76fm" event={"ID":"713aba87-e440-4d2f-a339-52b3f222fa97","Type":"ContainerStarted","Data":"0f1d9b7623f79f537e0583c362f333dead9b34cd227f2a6e7abec4c7d99b780e"}
Nov 02 22:04:49.248938 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.248868    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 21:59:47 +0000 UTC" deadline="2028-06-29 10:45:17.164299446 +0000 UTC"
Nov 02 22:04:49.248938 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:49.248935    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14508h40m27.915369162s"
Nov 02 22:04:50.865622 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:50.865561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:50.866139 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:50.865663    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:50.866139 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:50.865833    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:04:50.866139 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:50.865904    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs podName:56b1aa47-8106-416e-869d-bfc5c10ef795 nodeName:}" failed. No retries permitted until 2026-11-02 22:04:54.86588274 +0000 UTC m=+9.405327554 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs") pod "network-metrics-daemon-69fj9" (UID: "56b1aa47-8106-416e-869d-bfc5c10ef795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:04:50.866139 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:50.865977    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 22:04:50.866139 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:50.866013    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret podName:e47f56ea-3a73-4ff0-965d-3a21319aef2e nodeName:}" failed. No retries permitted until 2026-11-02 22:04:54.866002805 +0000 UTC m=+9.405447616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret") pod "global-pull-secret-syncer-78snq" (UID: "e47f56ea-3a73-4ff0-965d-3a21319aef2e") : object "kube-system"/"original-pull-secret" not registered
Nov 02 22:04:50.966091 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:50.966053    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:50.966286 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:50.966254    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 22:04:50.966286 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:50.966278    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 22:04:50.966402 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:50.966292    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dmjvr for pod openshift-network-diagnostics/network-check-target-7j964: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:04:50.966402 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:50.966370    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr podName:3505867a-1caf-4cef-b570-841afd5f7d85 nodeName:}" failed. No retries permitted until 2026-11-02 22:04:54.966350915 +0000 UTC m=+9.505795735 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmjvr" (UniqueName: "kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr") pod "network-check-target-7j964" (UID: "3505867a-1caf-4cef-b570-841afd5f7d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:04:51.093776 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:51.093253    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:51.093776 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:51.093402    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:04:51.093776 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:51.093489    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:51.093776 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:51.093562    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:04:51.093776 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:51.093640    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:51.093776 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:51.093720    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:04:53.093296 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:53.093217    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:53.093924 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:53.093362    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:04:53.093924 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:53.093457    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:53.093924 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:53.093529    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:04:53.093924 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:53.093609    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:53.093924 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:53.093703    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:04:53.148291 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:53.148237    2583 generic.go:356] "Generic (PLEG): container finished" podID="657ace375fa97dc1e56f2b6e64a6ac61" containerID="3adf846d77093c82ddc07986ec525030916968fcb2ccfa72c431fc621a816505" exitCode=0
Nov 02 22:04:53.148467 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:53.148355    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal" event={"ID":"657ace375fa97dc1e56f2b6e64a6ac61","Type":"ContainerDied","Data":"3adf846d77093c82ddc07986ec525030916968fcb2ccfa72c431fc621a816505"}
Nov 02 22:04:53.154473 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:53.154440    2583 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="b64b7225e9c97e1d9796f652d6f3ee43af837b0a80ea6fc146fdb4845b734ff1" exitCode=0
Nov 02 22:04:53.154608 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:53.154546    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbz67" event={"ID":"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4","Type":"ContainerDied","Data":"b64b7225e9c97e1d9796f652d6f3ee43af837b0a80ea6fc146fdb4845b734ff1"}
Nov 02 22:04:53.160834 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:53.160387    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal" event={"ID":"376e7bd75c236d364d4749b5f447d5f4","Type":"ContainerStarted","Data":"4eea97239249f65ac4c30eea9101948bec459506cac63d3694472b9ce5cc40c9"}
Nov 02 22:04:53.518769 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:53.518374    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal" podStartSLOduration=6.518357913 podStartE2EDuration="6.518357913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:53.391747644 +0000 UTC m=+7.931192465" watchObservedRunningTime="2026-11-02 22:04:53.518357913 +0000 UTC m=+8.057802732"
Nov 02 22:04:54.170765 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:54.170674    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal" event={"ID":"657ace375fa97dc1e56f2b6e64a6ac61","Type":"ContainerStarted","Data":"3ab2fbc96ec4a60b5ab9020c9cae0340c55e96d81120f5ab6e5f61042f525819"}
Nov 02 22:04:54.175120 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:54.175078    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ww4zl" event={"ID":"4843105e-92f4-4b83-83fd-06e7a217e009","Type":"ContainerStarted","Data":"4af549d393e351a2b9c3d51f1ddad6fc6f1aaaeaa46a9182c46dc422851c3a20"}
Nov 02 22:04:54.410241 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:54.410142    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal" podStartSLOduration=7.410121044 podStartE2EDuration="7.410121044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:54.291334138 +0000 UTC m=+8.830778958" watchObservedRunningTime="2026-11-02 22:04:54.410121044 +0000 UTC m=+8.949565865"
Nov 02 22:04:54.904978 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:54.904935    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:54.905148 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:54.905041    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:54.905148 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:54.905135    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:04:54.905239 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:54.905149    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 22:04:54.905239 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:54.905216    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs podName:56b1aa47-8106-416e-869d-bfc5c10ef795 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:02.905194308 +0000 UTC m=+17.444639120 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs") pod "network-metrics-daemon-69fj9" (UID: "56b1aa47-8106-416e-869d-bfc5c10ef795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:04:54.905322 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:54.905242    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret podName:e47f56ea-3a73-4ff0-965d-3a21319aef2e nodeName:}" failed. No retries permitted until 2026-11-02 22:05:02.905227534 +0000 UTC m=+17.444672335 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret") pod "global-pull-secret-syncer-78snq" (UID: "e47f56ea-3a73-4ff0-965d-3a21319aef2e") : object "kube-system"/"original-pull-secret" not registered
Nov 02 22:04:55.006829 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:55.006315    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:55.006829 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:55.006548    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 22:04:55.006829 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:55.006572    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 22:04:55.006829 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:55.006585    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dmjvr for pod openshift-network-diagnostics/network-check-target-7j964: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:04:55.006829 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:55.006651    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr podName:3505867a-1caf-4cef-b570-841afd5f7d85 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:03.006631489 +0000 UTC m=+17.546076309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmjvr" (UniqueName: "kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr") pod "network-check-target-7j964" (UID: "3505867a-1caf-4cef-b570-841afd5f7d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:04:55.093650 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:55.093543    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:55.093814 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:55.093674    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:04:55.093814 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:55.093747    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:55.093903 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:55.093827    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:04:55.093903 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:55.093873    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:55.093974 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:55.093936    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:04:55.180475 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:55.180438    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v76fm" event={"ID":"713aba87-e440-4d2f-a339-52b3f222fa97","Type":"ContainerStarted","Data":"af44b2c7f443035c6e8216b2a17720bc29eb5bd96827a0f07af65eef54aa15e5"}
Nov 02 22:04:55.289280 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:55.289214    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ww4zl" podStartSLOduration=3.171541837 podStartE2EDuration="8.28919493s" totalImagesPullingTime="5.117653093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:54.41117379 +0000 UTC m=+8.950618612" watchObservedRunningTime="2026-11-02 22:04:55.28919493 +0000 UTC m=+9.828639748"
Nov 02 22:04:55.850149 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:55.850119    2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:04:55.850612 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:55.850587    2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:04:55.960941 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:55.960884    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v76fm" podStartSLOduration=2.6389949919999998 podStartE2EDuration="8.960862875s" totalImagesPullingTime="6.321867883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:55.290125644 +0000 UTC m=+9.829570462" watchObservedRunningTime="2026-11-02 22:04:55.960862875 +0000 UTC m=+10.500307691"
Nov 02 22:04:56.186585 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:56.186507    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr" event={"ID":"6b703071-5807-41b5-afe0-a2c85e23af47","Type":"ContainerStarted","Data":"468f02edd34f73b91cdb3b864133c9116da015f61853ccde909dc185363d3813"}
Nov 02 22:04:57.094839 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:57.094146    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:57.094839 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:57.094501    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:04:57.094839 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:57.094564    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:57.094839 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:57.094637    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:04:57.094839 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:57.094680    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:57.094839 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:57.094734    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:04:57.188908 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:57.188406    2583 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 22:04:57.731088 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:57.730967    2583 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 22:04:58.099108 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:58.098996    2583 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T22:04:57.730983854Z","UUID":"6e397042-d77c-4beb-94ca-fd6157ed4772","Handler":null,"Name":"","Endpoint":""}
Nov 02 22:04:58.102248 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:58.102227    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 22:04:58.102395 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:58.102259    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 22:04:58.193088 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:58.193056    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr" event={"ID":"6b703071-5807-41b5-afe0-a2c85e23af47","Type":"ContainerStarted","Data":"28b801187ed9e9a663f8ae7720222824258d93184be4d49c8cec7baf84fef528"}
Nov 02 22:04:58.194920 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:58.194891    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dkbhl" event={"ID":"0caf91c1-266c-405f-92b4-070981db24c3","Type":"ContainerStarted","Data":"049bef685244dae2a8bb829b70b1ec55b2e27a338a86381b82fc0581410860b9"}
Nov 02 22:04:58.197271 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:58.196925    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t5mqk" event={"ID":"21f54242-5bab-4162-a435-85a080f8685c","Type":"ContainerStarted","Data":"87887b927d01de2b71f0d4452c677c32a295a6a9a6b30ba393cae1cb682a98c6"}
Nov 02 22:04:58.309477 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:58.309399    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-dkbhl" podStartSLOduration=2.494709335 podStartE2EDuration="11.309384238s" totalImagesPullingTime="8.814674903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:58.307832249 +0000 UTC m=+12.847277064" watchObservedRunningTime="2026-11-02 22:04:58.309384238 +0000 UTC m=+12.848829053"
Nov 02 22:04:58.417917 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:58.417856    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-t5mqk" podStartSLOduration=2.9197281459999997 podStartE2EDuration="12.417835123s" totalImagesPullingTime="9.498106977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:04:58.417173628 +0000 UTC m=+12.956618445" watchObservedRunningTime="2026-11-02 22:04:58.417835123 +0000 UTC m=+12.957279940"
Nov 02 22:04:59.093975 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:59.093934    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:04:59.094157 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:59.093955    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:04:59.094157 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:59.094068    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:04:59.094157 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:59.094085    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:04:59.094338 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:59.094181    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:04:59.094338 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:04:59.094265    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:04:59.201471 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:59.201433    2583 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="8abb0d692616ce8d49943ab3c4fb7ee09e6a9878ba90d6312167de10b0070bb9" exitCode=0
Nov 02 22:04:59.201961 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:04:59.201524    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbz67" event={"ID":"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4","Type":"ContainerDied","Data":"8abb0d692616ce8d49943ab3c4fb7ee09e6a9878ba90d6312167de10b0070bb9"}
Nov 02 22:05:00.205831 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:00.205583    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bpxkm" event={"ID":"1e801619-9294-420b-8ad5-cd1005832f16","Type":"ContainerStarted","Data":"d2b598f7ceaf93db3a6a818815409c1bce75203a87282f2269cf2e5a0f1a2d4a"}
Nov 02 22:05:00.207562 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:00.207533    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f4n2z" event={"ID":"fc25374f-cb3c-45ce-a523-af6fa11b570e","Type":"ContainerStarted","Data":"8d5e989a86d9b61b4489fbacc9c4515f3dbab981943587d950bc5a0c13f53b2a"}
Nov 02 22:05:00.210399 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:00.210003    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr" event={"ID":"6b703071-5807-41b5-afe0-a2c85e23af47","Type":"ContainerStarted","Data":"26b794edecbf1c6fd1af04f70102821e08d08323487788c165ab498d81948228"}
Nov 02 22:05:00.433046 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:00.432991    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bpxkm" podStartSLOduration=5.188757571 podStartE2EDuration="14.4329623s" totalImagesPullingTime="9.244204729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:00.317815251 +0000 UTC m=+14.857260067" watchObservedRunningTime="2026-11-02 22:05:00.4329623 +0000 UTC m=+14.972407097"
Nov 02 22:05:00.433498 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:00.433453    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr" podStartSLOduration=3.156364687 podStartE2EDuration="13.433441042s" totalImagesPullingTime="10.277076355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:04:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:00.433118963 +0000 UTC m=+14.972563761" watchObservedRunningTime="2026-11-02 22:05:00.433441042 +0000 UTC m=+14.972885858"
Nov 02 22:05:00.546138 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:00.546084    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f4n2z" podStartSLOduration=2.573569188 podStartE2EDuration="13.546066139s" totalImagesPullingTime="10.972496951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:04:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:00.545535852 +0000 UTC m=+15.084980669" watchObservedRunningTime="2026-11-02 22:05:00.546066139 +0000 UTC m=+15.085510957"
Nov 02 22:05:01.093514 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:01.093486    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:01.093684 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:01.093593    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:01.093684 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:01.093602    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:05:01.093793 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:01.093696    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:05:01.093793 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:01.093760    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:01.093884 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:01.093827    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:05:01.214588 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:01.214555    2583 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="8367bd33fdd34ac4c521dc481bb88be969217b2b3437f96622740f8a9ba05378" exitCode=0
Nov 02 22:05:01.215078 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:01.214659    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbz67" event={"ID":"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4","Type":"ContainerDied","Data":"8367bd33fdd34ac4c521dc481bb88be969217b2b3437f96622740f8a9ba05378"}
Nov 02 22:05:02.814868 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:02.814794    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:05:02.815286 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:02.814936    2583 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 22:05:02.815450 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:02.815421    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:05:02.973977 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:02.973943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:02.974152 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:02.973994    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:02.974152 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:02.974127    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 22:05:02.974152 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:02.974127    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:05:02.974273 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:02.974202    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret podName:e47f56ea-3a73-4ff0-965d-3a21319aef2e nodeName:}" failed. No retries permitted until 2026-11-02 22:05:18.974183427 +0000 UTC m=+33.513628222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret") pod "global-pull-secret-syncer-78snq" (UID: "e47f56ea-3a73-4ff0-965d-3a21319aef2e") : object "kube-system"/"original-pull-secret" not registered
Nov 02 22:05:02.974273 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:02.974223    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs podName:56b1aa47-8106-416e-869d-bfc5c10ef795 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:18.97421362 +0000 UTC m=+33.513658437 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs") pod "network-metrics-daemon-69fj9" (UID: "56b1aa47-8106-416e-869d-bfc5c10ef795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 22:05:03.074700 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:03.074612    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:03.074860 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:03.074765    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 22:05:03.074860 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:03.074784    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 22:05:03.074860 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:03.074797    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dmjvr for pod openshift-network-diagnostics/network-check-target-7j964: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:05:03.074860 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:03.074855    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr podName:3505867a-1caf-4cef-b570-841afd5f7d85 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:19.074837584 +0000 UTC m=+33.614282394 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dmjvr" (UniqueName: "kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr") pod "network-check-target-7j964" (UID: "3505867a-1caf-4cef-b570-841afd5f7d85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 22:05:03.094022 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:03.093994    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:03.094169 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:03.094091    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:03.094169 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:03.094101    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:05:03.094269 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:03.094198    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:05:03.094269 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:03.094255    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:03.094397 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:03.094310    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:05:03.221212 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:03.221177    2583 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="7b4e58ea47dd6d78818f1082c951403e8007a2dbf1e796e6e0776cba78bd8f18" exitCode=0
Nov 02 22:05:03.221402 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:03.221268    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbz67" event={"ID":"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4","Type":"ContainerDied","Data":"7b4e58ea47dd6d78818f1082c951403e8007a2dbf1e796e6e0776cba78bd8f18"}
Nov 02 22:05:04.226469 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:04.226277    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx" event={"ID":"9cfdf849-3285-4c0c-bcc8-2401667a460b","Type":"ContainerStarted","Data":"ffa80024bc6b1b013309ec110390b4298b5467eba8491e6a56121660f3544f8c"}
Nov 02 22:05:04.227028 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:04.226522    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx" event={"ID":"9cfdf849-3285-4c0c-bcc8-2401667a460b","Type":"ContainerStarted","Data":"154c6786e4fe1bb7d10a86f2dc40df453467c715b3462f735dd540f3075a1670"}
Nov 02 22:05:04.227028 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:04.226570    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx" event={"ID":"9cfdf849-3285-4c0c-bcc8-2401667a460b","Type":"ContainerStarted","Data":"788f90d54732a2cea8e61b62a2179a79a1aa8fd8c6f61ce6003bd3c4fdd2ffba"}
Nov 02 22:05:04.227028 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:04.226609    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx" event={"ID":"9cfdf849-3285-4c0c-bcc8-2401667a460b","Type":"ContainerStarted","Data":"0925a5826205a91514aa6cc2d4feecc33f55a52e46e5727067810999a8dd77d5"}
Nov 02 22:05:04.227028 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:04.226652    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx" event={"ID":"9cfdf849-3285-4c0c-bcc8-2401667a460b","Type":"ContainerStarted","Data":"1e89ce3d7c2e9654add717331197049ef036749f6b7e316b4f3d2f7971e53c43"}
Nov 02 22:05:04.227028 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:04.226697    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx" event={"ID":"9cfdf849-3285-4c0c-bcc8-2401667a460b","Type":"ContainerStarted","Data":"91a8a4fe3f1f9b789bb418c221bf86d7f64e3f45f1f7a6a219b75cd9bfffb8cd"}
Nov 02 22:05:05.094035 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:05.094001    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:05.094035 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:05.094025    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:05.094255 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:05.094136    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:05:05.094255 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:05.094178    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:05:05.094255 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:05.094220    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:05.094400 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:05.094287    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:05:06.233467 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:06.233431    2583 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="c965d3d57831dadfa49959f7b95f850209de58db15bf2ed48495b67dedd319d2" exitCode=0
Nov 02 22:05:06.234058 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:06.233501    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbz67" event={"ID":"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4","Type":"ContainerDied","Data":"c965d3d57831dadfa49959f7b95f850209de58db15bf2ed48495b67dedd319d2"}
Nov 02 22:05:06.237428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:06.237396    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx" event={"ID":"9cfdf849-3285-4c0c-bcc8-2401667a460b","Type":"ContainerStarted","Data":"4497b94cc2c920c1b855a76c9619124e6983550260142870fd115647bae31714"}
Nov 02 22:05:07.093288 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:07.093258    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:07.093468 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:07.093336    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:07.093468 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:07.093437    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:07.093468 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:07.093446    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:05:07.093564 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:07.093526    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:05:07.093604 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:07.093589    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:05:07.242504 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:07.242475    2583 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="29ea596b952de5e75c9b14f35f5bd20a58dc54cb426621fb24778267ba59a97b" exitCode=0
Nov 02 22:05:07.242867 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:07.242510    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbz67" event={"ID":"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4","Type":"ContainerDied","Data":"29ea596b952de5e75c9b14f35f5bd20a58dc54cb426621fb24778267ba59a97b"}
Nov 02 22:05:08.248483 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:08.248276    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xbz67" event={"ID":"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4","Type":"ContainerStarted","Data":"bb71bf58098861eff618f9f1ea83525ca3561dffb7f89052b771eae5200fd9c8"}
Nov 02 22:05:08.428717 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:08.428667    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xbz67" podStartSLOduration=7.629591301 podStartE2EDuration="21.428652028s" totalImagesPullingTime="13.799060727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:04:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:08.428601524 +0000 UTC m=+22.968046340" watchObservedRunningTime="2026-11-02 22:05:08.428652028 +0000 UTC m=+22.968096837"
Nov 02 22:05:09.094212 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:09.094172    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:09.094212 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:09.094204    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:09.094481 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:09.094172    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:09.094481 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:09.094314    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:05:09.094481 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:09.094393    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:05:09.094481 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:09.094468    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:05:09.254285 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:09.254245    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx" event={"ID":"9cfdf849-3285-4c0c-bcc8-2401667a460b","Type":"ContainerStarted","Data":"58728d14af261169c87f4242f79ae2d2313b82a46b1953c3a9f947d546de6049"}
Nov 02 22:05:09.520761 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:09.520709    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx" podStartSLOduration=8.28050347 podStartE2EDuration="23.520695836s" totalImagesPullingTime="15.240192366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 22:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:09.520385923 +0000 UTC m=+24.059830739" watchObservedRunningTime="2026-11-02 22:05:09.520695836 +0000 UTC m=+24.060140649"
Nov 02 22:05:10.256305 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:10.256270    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:10.256751 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:10.256351    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:10.256751 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:10.256406    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:10.271821 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:10.271797    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:10.271958 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:10.271895    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:11.093807 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:11.093776    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:11.093807 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:11.093805    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:11.094005 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:11.093878    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:05:11.094005 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:11.093926    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:05:11.094005 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:11.093975    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:11.094102 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:11.094061    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:05:11.123172 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:11.123145    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-78snq"]
Nov 02 22:05:11.126559 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:11.126533    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7j964"]
Nov 02 22:05:11.129454 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:11.129435    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-69fj9"]
Nov 02 22:05:11.257798 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:11.257767    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:11.257798 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:11.257782    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:11.258244 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:11.257765    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:11.258244 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:11.257883    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:05:11.258244 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:11.257943    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:05:11.258351 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:11.258278    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:05:13.093824 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:13.093792    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:13.093824 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:13.093822    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:13.094355 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:13.093834    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:13.094355 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:13.093930    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:05:13.094355 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:13.094031    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:05:13.094355 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:13.094094    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:05:15.093767 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:15.093589    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:15.094270 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:15.093640    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:15.094270 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:15.093848    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-69fj9" podUID="56b1aa47-8106-416e-869d-bfc5c10ef795"
Nov 02 22:05:15.094270 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:15.093944    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7j964" podUID="3505867a-1caf-4cef-b570-841afd5f7d85"
Nov 02 22:05:15.094270 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:15.093640    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:15.094270 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:15.094040    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78snq" podUID="e47f56ea-3a73-4ff0-965d-3a21319aef2e"
Nov 02 22:05:16.227504 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.227481    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeReady"
Nov 02 22:05:16.227892 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.227607    2583 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 02 22:05:16.373215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.373185    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xmv5g"]
Nov 02 22:05:16.387574 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.387544    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"]
Nov 02 22:05:16.387746 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.387656    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xmv5g"
Nov 02 22:05:16.398652 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.398631    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-tjhsr"]
Nov 02 22:05:16.398802 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.398773    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:16.406642 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.406620    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-7vddh"]
Nov 02 22:05:16.406753 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.406702    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-tjhsr"
Nov 02 22:05:16.417953 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.417932    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-b448bfb57-lttxh"]
Nov 02 22:05:16.418061 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.418047    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-7vddh"
Nov 02 22:05:16.430188 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.430166    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56dc494c89-qz46b"]
Nov 02 22:05:16.430401 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.430359    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.442024 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.441750    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.442024 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.441761    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.444150 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.444130    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"]
Nov 02 22:05:16.444321 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.444304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.450394 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.450320    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 02 22:05:16.463532 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.463513    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"]
Nov 02 22:05:16.463633 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.463619    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.469121 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.469090    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:16.469231 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.469180    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j2wx\" (UniqueName: \"kubernetes.io/projected/101f50b9-bfd1-405a-ade4-6e1cc683dfda-kube-api-access-8j2wx\") pod \"kas-connection-checker-685895c7c9-tjhsr\" (UID: \"101f50b9-bfd1-405a-ade4-6e1cc683dfda\") " pod="kube-system/kas-connection-checker-685895c7c9-tjhsr"
Nov 02 22:05:16.469286 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.469227    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqjht\" (UniqueName: \"kubernetes.io/projected/8759d3be-8ddd-4719-808f-3611fcd7f887-kube-api-access-zqjht\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:16.469349 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.469328    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6zhx\" (UniqueName: \"kubernetes.io/projected/34ca0f75-5a97-4496-a7ae-060723b2d625-kube-api-access-s6zhx\") pod \"kas-connection-checker-685895c7c9-xmv5g\" (UID: \"34ca0f75-5a97-4496-a7ae-060723b2d625\") " pod="kube-system/kas-connection-checker-685895c7c9-xmv5g"
Nov 02 22:05:16.469417 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.469381    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8759d3be-8ddd-4719-808f-3611fcd7f887-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:16.473886 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.473867    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66b78b7d88-kdfn8"]
Nov 02 22:05:16.474009 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.473994    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:16.482888 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.482867    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-9tsq4"]
Nov 02 22:05:16.482992 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.482977    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.489946 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.489929    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-x67tb"]
Nov 02 22:05:16.490046 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.490031    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.500209 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.500187    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"]
Nov 02 22:05:16.500736 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.500465    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:16.503322 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.503302    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw"]
Nov 02 22:05:16.503456 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.503437    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.512778 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.510646    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4t246"]
Nov 02 22:05:16.512778 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.510785    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw"
Nov 02 22:05:16.519759 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.519737    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"]
Nov 02 22:05:16.519958 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.519940    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4t246"
Nov 02 22:05:16.526058 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.526037    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gd729"]
Nov 02 22:05:16.526151 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.526137    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.532298 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.532277    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9pqwd"]
Nov 02 22:05:16.532412 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.532329    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.538422 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538404    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xmv5g"]
Nov 02 22:05:16.538496 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538463    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"]
Nov 02 22:05:16.538496 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538489    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-7vddh"]
Nov 02 22:05:16.538562 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538509    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-tjhsr"]
Nov 02 22:05:16.538562 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538515    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:16.538562 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538528    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"]
Nov 02 22:05:16.538562 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538545    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"]
Nov 02 22:05:16.538687 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538569    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b448bfb57-lttxh"]
Nov 02 22:05:16.538687 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538589    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-9tsq4"]
Nov 02 22:05:16.538687 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538603    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"]
Nov 02 22:05:16.538687 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538615    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw"]
Nov 02 22:05:16.538687 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538628    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gd729"]
Nov 02 22:05:16.538687 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538649    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-x67tb"]
Nov 02 22:05:16.538687 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538667    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66b78b7d88-kdfn8"]
Nov 02 22:05:16.538976 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538696    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4t246"]
Nov 02 22:05:16.538976 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538754    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"]
Nov 02 22:05:16.538976 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.538776    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56dc494c89-qz46b"]
Nov 02 22:05:16.541769 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.541583    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.541769 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.541639    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.541931 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.541834    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.541984 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.541898    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.542145 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.542100    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.542347 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.542328    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 02 22:05:16.542450 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.542421    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.542508 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.542483    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 02 22:05:16.542589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.542563    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.542589 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.542583    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.542727 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.542652    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 02 22:05:16.542727 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.542688    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 02 22:05:16.542727 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.542729    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.542866 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.542764    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.570525 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.570496    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9fkg\" (UniqueName: \"kubernetes.io/projected/dde46c9c-2501-4545-918c-9a3af3a6b381-kube-api-access-w9fkg\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.570525 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.570527    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-tmp\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.570710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.570552    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:16.570710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.570568    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-default-certificate\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.570710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.570646    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.570710 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.570701    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:16.570862 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.570754    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-registry-certificates\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.570862 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.570801    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls podName:8759d3be-8ddd-4719-808f-3611fcd7f887 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:17.070777464 +0000 UTC m=+31.610222281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-rjgqf" (UID: "8759d3be-8ddd-4719-808f-3611fcd7f887") : secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:16.570945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.570871    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f32e923-7f5f-4ba7-b44d-6441fc46f876-serving-cert\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.570945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.570918    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:16.571039 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.570944    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.571039 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571004    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-trusted-ca\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.571152 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571051    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x47x\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-kube-api-access-7x47x\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.571152 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571071    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/624e4d57-0db4-4be3-b512-e5065d800d01-nginx-conf\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:16.571152 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571086    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-config\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.571252 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571157    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:16.571252 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571194    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7wvw\" (UniqueName: \"kubernetes.io/projected/daa20bd0-d6e1-4044-812a-37ef99a19dac-kube-api-access-q7wvw\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.571252 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571220    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-service-ca-bundle\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.571428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571325    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b66dw\" (UniqueName: \"kubernetes.io/projected/3465159d-8024-48a2-8408-eb1d1db96a8a-kube-api-access-b66dw\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.571428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571356    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-image-registry-private-configuration\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.571428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571406    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-bound-sa-token\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.571566 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571431    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25h79\" (UniqueName: \"kubernetes.io/projected/8f32e923-7f5f-4ba7-b44d-6441fc46f876-kube-api-access-25h79\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.571566 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571454    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6zhx\" (UniqueName: \"kubernetes.io/projected/34ca0f75-5a97-4496-a7ae-060723b2d625-kube-api-access-s6zhx\") pod \"kas-connection-checker-685895c7c9-xmv5g\" (UID: \"34ca0f75-5a97-4496-a7ae-060723b2d625\") " pod="kube-system/kas-connection-checker-685895c7c9-xmv5g"
Nov 02 22:05:16.571566 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571484    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-trusted-ca\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.571697 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571565    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/daa20bd0-d6e1-4044-812a-37ef99a19dac-serving-cert\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.571697 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-snapshots\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.571697 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8759d3be-8ddd-4719-808f-3611fcd7f887-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:16.571844 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571709    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-certificates\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.571844 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571758    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-installation-pull-secrets\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.571844 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571797    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j2wx\" (UniqueName: \"kubernetes.io/projected/101f50b9-bfd1-405a-ade4-6e1cc683dfda-kube-api-access-8j2wx\") pod \"kas-connection-checker-685895c7c9-tjhsr\" (UID: \"101f50b9-bfd1-405a-ade4-6e1cc683dfda\") " pod="kube-system/kas-connection-checker-685895c7c9-tjhsr"
Nov 02 22:05:16.571946 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571869    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqjht\" (UniqueName: \"kubernetes.io/projected/8759d3be-8ddd-4719-808f-3611fcd7f887-kube-api-access-zqjht\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:16.571946 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571917    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/daa20bd0-d6e1-4044-812a-37ef99a19dac-config\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.572035 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571945    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-stats-auth\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.572035 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.571968    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.572035 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572027    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-bound-sa-token\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.572167 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572054    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3465159d-8024-48a2-8408-eb1d1db96a8a-serving-cert\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.572167 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572078    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dx64\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-kube-api-access-2dx64\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.572167 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572122    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-image-registry-private-configuration\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.572167 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572143    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e6866c0-b229-4cd8-b977-32d366cced05-ca-trust-extracted\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.572328 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572178    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-installation-pull-secrets\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.572328 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572203    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfbdb\" (UniqueName: \"kubernetes.io/projected/012f51ba-d2ba-4d6b-b90d-67fe89601119-kube-api-access-sfbdb\") pod \"kas-connection-checker-685895c7c9-7vddh\" (UID: \"012f51ba-d2ba-4d6b-b90d-67fe89601119\") " pod="kube-system/kas-connection-checker-685895c7c9-7vddh"
Nov 02 22:05:16.572328 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572236    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:16.572328 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572286    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8b9eeac-3e3f-4612-a696-2ede201d83ac-ca-trust-extracted\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.572328 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572309    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.572328 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572324    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-trusted-ca\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.572594 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.572594 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.572400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfdlq\" (UniqueName: \"kubernetes.io/projected/35ec85a5-34bf-4cd2-b209-7744c6f51a20-kube-api-access-lfdlq\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:16.590491 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.590467    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 02 22:05:16.593445 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.593429    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8759d3be-8ddd-4719-808f-3611fcd7f887-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:16.594177 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.594158    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 02 22:05:16.597731 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.597713    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 02 22:05:16.597783 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.597759    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 02 22:05:16.600846 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.600778    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 02 22:05:16.600846 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.600798    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 02 22:05:16.600846 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.600845    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 02 22:05:16.601008 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.600855    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 02 22:05:16.601151 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.601130    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jtvd2\""
Nov 02 22:05:16.601206 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.601194    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 02 22:05:16.601643 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.601627    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9pqwd"]
Nov 02 22:05:16.602914 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.602898    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fkc8k\""
Nov 02 22:05:16.602971 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.602902    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 02 22:05:16.603215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603171    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-89mts\""
Nov 02 22:05:16.603215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603178    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 02 22:05:16.603215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603190    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 02 22:05:16.603215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603201    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 02 22:05:16.603428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603234    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 02 22:05:16.603428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603238    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 02 22:05:16.603428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603284    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-xh86t\""
Nov 02 22:05:16.603428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603333    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 02 22:05:16.603428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603351    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-l6kn9\""
Nov 02 22:05:16.603428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603392    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t4sw7\""
Nov 02 22:05:16.603428 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603412    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v45jr\""
Nov 02 22:05:16.603625 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603472    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gwrjd\""
Nov 02 22:05:16.603625 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.603522    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-98gpk\""
Nov 02 22:05:16.640564 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.640534    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6zhx\" (UniqueName: \"kubernetes.io/projected/34ca0f75-5a97-4496-a7ae-060723b2d625-kube-api-access-s6zhx\") pod \"kas-connection-checker-685895c7c9-xmv5g\" (UID: \"34ca0f75-5a97-4496-a7ae-060723b2d625\") " pod="kube-system/kas-connection-checker-685895c7c9-xmv5g"
Nov 02 22:05:16.640708 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.640683    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j2wx\" (UniqueName: \"kubernetes.io/projected/101f50b9-bfd1-405a-ade4-6e1cc683dfda-kube-api-access-8j2wx\") pod \"kas-connection-checker-685895c7c9-tjhsr\" (UID: \"101f50b9-bfd1-405a-ade4-6e1cc683dfda\") " pod="kube-system/kas-connection-checker-685895c7c9-tjhsr"
Nov 02 22:05:16.640756 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.640743    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqjht\" (UniqueName: \"kubernetes.io/projected/8759d3be-8ddd-4719-808f-3611fcd7f887-kube-api-access-zqjht\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:16.642668 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642647    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.642668 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642657    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.642668 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642667    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.642891 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642809    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.642891 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642852    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.642891 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642876    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 02 22:05:16.642891 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642883    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.642891 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642891    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.643125 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642884    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.643125 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642910    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:16.643125 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642917    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:16.643125 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642919    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 22:05:16.643125 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642911    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 02 22:05:16.643125 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.642938    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 02 22:05:16.650440 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.650424    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 22:05:16.650693 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.650681    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 22:05:16.650763 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.650750    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-4gtmj\""
Nov 02 22:05:16.672963 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.672941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.672963 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.672967    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-trusted-ca\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.673142 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.672991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.673142 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfdlq\" (UniqueName: \"kubernetes.io/projected/35ec85a5-34bf-4cd2-b209-7744c6f51a20-kube-api-access-lfdlq\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:16.673142 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.673098    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:16.673142 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.673112    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b78b7d88-kdfn8: secret "image-registry-tls" not found
Nov 02 22:05:16.673142 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9fkg\" (UniqueName: \"kubernetes.io/projected/dde46c9c-2501-4545-918c-9a3af3a6b381-kube-api-access-w9fkg\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.673415 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.673160    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls podName:a8b9eeac-3e3f-4612-a696-2ede201d83ac nodeName:}" failed. No retries permitted until 2026-11-02 22:05:17.173144942 +0000 UTC m=+31.712589736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls") pod "image-registry-66b78b7d88-kdfn8" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac") : secret "image-registry-tls" not found
Nov 02 22:05:16.673415 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673188    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-tmp\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.673415 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-default-certificate\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.673415 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.673415 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673282    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-registry-certificates\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.673415 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f32e923-7f5f-4ba7-b44d-6441fc46f876-serving-cert\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.673415 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673340    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:16.673415 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673384    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.673415 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.673416    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:17.173399625 +0000 UTC m=+31.712844433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : configmap references non-existent config key: service-ca.crt
Nov 02 22:05:16.673844 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673435    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-trusted-ca\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.673844 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.673460    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:16.673844 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfaa3942-3844-475e-9d7a-95bf64101115-tmp-dir\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.673844 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x47x\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-kube-api-access-7x47x\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.673844 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673553    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-tmp\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.673844 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673597    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/624e4d57-0db4-4be3-b512-e5065d800d01-nginx-conf\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:16.673844 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.673471    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56dc494c89-qz46b: secret "image-registry-tls" not found
Nov 02 22:05:16.673844 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.673574    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 22:05:16.673844 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673639    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-config\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.673844 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.673747    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls podName:35ec85a5-34bf-4cd2-b209-7744c6f51a20 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:17.173730714 +0000 UTC m=+31.713175529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-xf2zt" (UID: "35ec85a5-34bf-4cd2-b209-7744c6f51a20") : secret "samples-operator-tls" not found
Nov 02 22:05:16.674309 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.673979    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-trusted-ca\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.674309 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674043    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:16.674309 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674076    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7wvw\" (UniqueName: \"kubernetes.io/projected/daa20bd0-d6e1-4044-812a-37ef99a19dac-kube-api-access-q7wvw\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.674309 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674101    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.674309 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674104    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-service-ca-bundle\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.674309 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674181    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-registry-certificates\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.674309 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.674309 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674245    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2t4v\" (UniqueName: \"kubernetes.io/projected/2dc1992e-41ec-4bde-9211-47162c8fb3c2-kube-api-access-t2t4v\") pod \"volume-data-source-validator-58c9f5f46d-xptqw\" (UID: \"2dc1992e-41ec-4bde-9211-47162c8fb3c2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674328    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfaa3942-3844-475e-9d7a-95bf64101115-config-volume\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.674399    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls podName:1e6866c0-b229-4cd8-b977-32d366cced05 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:17.174359242 +0000 UTC m=+31.713804056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls") pod "image-registry-56dc494c89-qz46b" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05") : secret "image-registry-tls" not found
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674425    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gls6g\" (UniqueName: \"kubernetes.io/projected/f020dd86-76b6-4433-a902-632d15269b91-kube-api-access-gls6g\") pod \"ingress-canary-9pqwd\" (UID: \"f020dd86-76b6-4433-a902-632d15269b91\") " pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674475    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b66dw\" (UniqueName: \"kubernetes.io/projected/3465159d-8024-48a2-8408-eb1d1db96a8a-kube-api-access-b66dw\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674475    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/624e4d57-0db4-4be3-b512-e5065d800d01-nginx-conf\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674528    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-image-registry-private-configuration\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674553    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-trusted-ca\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674563    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-bound-sa-token\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25h79\" (UniqueName: \"kubernetes.io/projected/8f32e923-7f5f-4ba7-b44d-6441fc46f876-kube-api-access-25h79\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674618    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-trusted-ca\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674642    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/daa20bd0-d6e1-4044-812a-37ef99a19dac-serving-cert\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-snapshots\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.674710 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674711    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ltdh\" (UniqueName: \"kubernetes.io/projected/bfaa3942-3844-475e-9d7a-95bf64101115-kube-api-access-7ltdh\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-certificates\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-installation-pull-secrets\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674807    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stbfv\" (UniqueName: \"kubernetes.io/projected/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-kube-api-access-stbfv\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674836    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-config\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674843    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f020dd86-76b6-4433-a902-632d15269b91-cert\") pod \"ingress-canary-9pqwd\" (UID: \"f020dd86-76b6-4433-a902-632d15269b91\") " pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674856    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.674979    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/daa20bd0-d6e1-4044-812a-37ef99a19dac-config\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675006    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-stats-auth\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675032    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-bound-sa-token\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675086    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqt4z\" (UniqueName: \"kubernetes.io/projected/2c712bb5-ab02-4ae7-8d21-b2274448ccf3-kube-api-access-cqt4z\") pod \"network-check-source-6f78d59f99-4t246\" (UID: \"2c712bb5-ab02-4ae7-8d21-b2274448ccf3\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4t246"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675110    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675139    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3465159d-8024-48a2-8408-eb1d1db96a8a-serving-cert\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675168    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dx64\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-kube-api-access-2dx64\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.675333 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675211    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-image-registry-private-configuration\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.676047 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675241    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e6866c0-b229-4cd8-b977-32d366cced05-ca-trust-extracted\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.676047 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675282    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-installation-pull-secrets\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.676047 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675313    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfbdb\" (UniqueName: \"kubernetes.io/projected/012f51ba-d2ba-4d6b-b90d-67fe89601119-kube-api-access-sfbdb\") pod \"kas-connection-checker-685895c7c9-7vddh\" (UID: \"012f51ba-d2ba-4d6b-b90d-67fe89601119\") " pod="kube-system/kas-connection-checker-685895c7c9-7vddh"
Nov 02 22:05:16.676047 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:16.676047 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8b9eeac-3e3f-4612-a696-2ede201d83ac-ca-trust-extracted\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.676047 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675394    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-snapshots\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.676047 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675413    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.676047 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675599    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-trusted-ca\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.676047 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.675908    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-certificates\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.676486 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.676068    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/daa20bd0-d6e1-4044-812a-37ef99a19dac-config\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.676486 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.676207    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f32e923-7f5f-4ba7-b44d-6441fc46f876-serving-cert\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.676486 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.676323    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 22:05:16.676486 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.676336    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 22:05:16.676486 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.676393    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:17.176357952 +0000 UTC m=+31.715802765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : secret "router-metrics-certs-default" not found
Nov 02 22:05:16.676486 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.676416    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert podName:624e4d57-0db4-4be3-b512-e5065d800d01 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:17.17640305 +0000 UTC m=+31.715847844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-x67tb" (UID: "624e4d57-0db4-4be3-b512-e5065d800d01") : secret "networking-console-plugin-cert" not found
Nov 02 22:05:16.676779 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.676686    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e6866c0-b229-4cd8-b977-32d366cced05-ca-trust-extracted\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.677392 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.676879    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8b9eeac-3e3f-4612-a696-2ede201d83ac-ca-trust-extracted\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.677392 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.677342    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/daa20bd0-d6e1-4044-812a-37ef99a19dac-serving-cert\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.677392 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.677342    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-service-ca-bundle\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.677862 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.677686    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-image-registry-private-configuration\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.678051 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.677964    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-default-certificate\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.678781 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.678756    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-image-registry-private-configuration\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.678926 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.678909    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-installation-pull-secrets\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.679184 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.679166    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3465159d-8024-48a2-8408-eb1d1db96a8a-serving-cert\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.679590 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.679569    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-stats-auth\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.680114 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.680098    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-installation-pull-secrets\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.707523 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.707505    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xmv5g"
Nov 02 22:05:16.730390 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.730360    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-tjhsr"
Nov 02 22:05:16.737151 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.737089    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25h79\" (UniqueName: \"kubernetes.io/projected/8f32e923-7f5f-4ba7-b44d-6441fc46f876-kube-api-access-25h79\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.737151 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.737143    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b66dw\" (UniqueName: \"kubernetes.io/projected/3465159d-8024-48a2-8408-eb1d1db96a8a-kube-api-access-b66dw\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.737780 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.737614    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-bound-sa-token\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.737780 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.737734    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfbdb\" (UniqueName: \"kubernetes.io/projected/012f51ba-d2ba-4d6b-b90d-67fe89601119-kube-api-access-sfbdb\") pod \"kas-connection-checker-685895c7c9-7vddh\" (UID: \"012f51ba-d2ba-4d6b-b90d-67fe89601119\") " pod="kube-system/kas-connection-checker-685895c7c9-7vddh"
Nov 02 22:05:16.737780 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.737747    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x47x\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-kube-api-access-7x47x\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.737958 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.737786    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7wvw\" (UniqueName: \"kubernetes.io/projected/daa20bd0-d6e1-4044-812a-37ef99a19dac-kube-api-access-q7wvw\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.737995 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.737966    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dx64\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-kube-api-access-2dx64\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:16.738078 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.738058    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-bound-sa-token\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:16.738215 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.738201    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9fkg\" (UniqueName: \"kubernetes.io/projected/dde46c9c-2501-4545-918c-9a3af3a6b381-kube-api-access-w9fkg\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:16.739709 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.739688    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfdlq\" (UniqueName: \"kubernetes.io/projected/35ec85a5-34bf-4cd2-b209-7744c6f51a20-kube-api-access-lfdlq\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:16.744526 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.744505    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-7vddh"
Nov 02 22:05:16.775810 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.775783    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqt4z\" (UniqueName: \"kubernetes.io/projected/2c712bb5-ab02-4ae7-8d21-b2274448ccf3-kube-api-access-cqt4z\") pod \"network-check-source-6f78d59f99-4t246\" (UID: \"2c712bb5-ab02-4ae7-8d21-b2274448ccf3\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4t246"
Nov 02 22:05:16.775902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.775817    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.775902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.775858    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.775992 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.775922    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfaa3942-3844-475e-9d7a-95bf64101115-tmp-dir\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.775992 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.775948    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.775992 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.775974    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2t4v\" (UniqueName: \"kubernetes.io/projected/2dc1992e-41ec-4bde-9211-47162c8fb3c2-kube-api-access-t2t4v\") pod \"volume-data-source-validator-58c9f5f46d-xptqw\" (UID: \"2dc1992e-41ec-4bde-9211-47162c8fb3c2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw"
Nov 02 22:05:16.776131 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.776002    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfaa3942-3844-475e-9d7a-95bf64101115-config-volume\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.776131 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.776028    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gls6g\" (UniqueName: \"kubernetes.io/projected/f020dd86-76b6-4433-a902-632d15269b91-kube-api-access-gls6g\") pod \"ingress-canary-9pqwd\" (UID: \"f020dd86-76b6-4433-a902-632d15269b91\") " pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:16.776131 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.776081    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ltdh\" (UniqueName: \"kubernetes.io/projected/bfaa3942-3844-475e-9d7a-95bf64101115-kube-api-access-7ltdh\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.776131 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.776120    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stbfv\" (UniqueName: \"kubernetes.io/projected/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-kube-api-access-stbfv\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.776319 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.776153    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f020dd86-76b6-4433-a902-632d15269b91-cert\") pod \"ingress-canary-9pqwd\" (UID: \"f020dd86-76b6-4433-a902-632d15269b91\") " pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:16.776319 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.776283    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfaa3942-3844-475e-9d7a-95bf64101115-tmp-dir\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.776458 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.776349    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 22:05:16.776458 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:16.776438    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls podName:bfaa3942-3844-475e-9d7a-95bf64101115 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:17.276419767 +0000 UTC m=+31.815864571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls") pod "dns-default-gd729" (UID: "bfaa3942-3844-475e-9d7a-95bf64101115") : secret "dns-default-metrics-tls" not found
Nov 02 22:05:16.776681 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.776656    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfaa3942-3844-475e-9d7a-95bf64101115-config-volume\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.777017 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.776996    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.778666 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.778646    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.778854 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.778835    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f020dd86-76b6-4433-a902-632d15269b91-cert\") pod \"ingress-canary-9pqwd\" (UID: \"f020dd86-76b6-4433-a902-632d15269b91\") " pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:16.780219 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.780203    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:16.813471 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.813416    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:16.835706 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.829516    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:16.841211 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.841140    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gls6g\" (UniqueName: \"kubernetes.io/projected/f020dd86-76b6-4433-a902-632d15269b91-kube-api-access-gls6g\") pod \"ingress-canary-9pqwd\" (UID: \"f020dd86-76b6-4433-a902-632d15269b91\") " pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:16.841857 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.841801    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2t4v\" (UniqueName: \"kubernetes.io/projected/2dc1992e-41ec-4bde-9211-47162c8fb3c2-kube-api-access-t2t4v\") pod \"volume-data-source-validator-58c9f5f46d-xptqw\" (UID: \"2dc1992e-41ec-4bde-9211-47162c8fb3c2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw"
Nov 02 22:05:16.841965 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.841891    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqt4z\" (UniqueName: \"kubernetes.io/projected/2c712bb5-ab02-4ae7-8d21-b2274448ccf3-kube-api-access-cqt4z\") pod \"network-check-source-6f78d59f99-4t246\" (UID: \"2c712bb5-ab02-4ae7-8d21-b2274448ccf3\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4t246"
Nov 02 22:05:16.842548 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.842510    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stbfv\" (UniqueName: \"kubernetes.io/projected/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-kube-api-access-stbfv\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.845433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.845410    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ltdh\" (UniqueName: \"kubernetes.io/projected/bfaa3942-3844-475e-9d7a-95bf64101115-kube-api-access-7ltdh\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:16.847995 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.847507    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:16.878472 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:16.877289    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:17.079608 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.079500    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:17.079769 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.079658    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:17.079769 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.079748    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls podName:8759d3be-8ddd-4719-808f-3611fcd7f887 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:18.079729525 +0000 UTC m=+32.619174326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-rjgqf" (UID: "8759d3be-8ddd-4719-808f-3611fcd7f887") : secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:17.093510 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.093479    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:17.093622 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.093547    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:17.093655 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.093631    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:17.132409 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.132386    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw"
Nov 02 22:05:17.139398 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.139377    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4t246"
Nov 02 22:05:17.144983 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.144954    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-tjhsr"]
Nov 02 22:05:17.148398 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.147109    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-6jxx8\""
Nov 02 22:05:17.148398 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.147265    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l4knj\""
Nov 02 22:05:17.148398 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.147358    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-g764w\""
Nov 02 22:05:17.148398 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.147476    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 22:05:17.148398 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.147109    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 22:05:17.148885 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:17.148861    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod101f50b9_bfd1_405a_ade4_6e1cc683dfda.slice/crio-3903975a55aacf078d9a316e1fe89ad9a10f24a1608e6425a2fade2399c2d77e WatchSource:0}: Error finding container 3903975a55aacf078d9a316e1fe89ad9a10f24a1608e6425a2fade2399c2d77e: Status 404 returned error can't find the container with id 3903975a55aacf078d9a316e1fe89ad9a10f24a1608e6425a2fade2399c2d77e
Nov 02 22:05:17.149071 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.149051    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xmv5g"]
Nov 02 22:05:17.154453 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:17.153591    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ca0f75_5a97_4496_a7ae_060723b2d625.slice/crio-06840d7e46923a329bc487595c4b2397af3eb9999df2376a55adb9c57bf44261 WatchSource:0}: Error finding container 06840d7e46923a329bc487595c4b2397af3eb9999df2376a55adb9c57bf44261: Status 404 returned error can't find the container with id 06840d7e46923a329bc487595c4b2397af3eb9999df2376a55adb9c57bf44261
Nov 02 22:05:17.167016 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.166949    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-7vddh"]
Nov 02 22:05:17.179143 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:17.179106    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012f51ba_d2ba_4d6b_b90d_67fe89601119.slice/crio-412cf88dada49b4ac00e82c9b197b7ae38c33a4e5ef99f4ccac1d9ba6d078a3c WatchSource:0}: Error finding container 412cf88dada49b4ac00e82c9b197b7ae38c33a4e5ef99f4ccac1d9ba6d078a3c: Status 404 returned error can't find the container with id 412cf88dada49b4ac00e82c9b197b7ae38c33a4e5ef99f4ccac1d9ba6d078a3c
Nov 02 22:05:17.180640 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.180618    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:17.180721 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.180670    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:17.180721 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.180717    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:17.180827 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.180745    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:17.180827 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.180778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:17.180827 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.180802    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 22:05:17.180972 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.180876    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert podName:624e4d57-0db4-4be3-b512-e5065d800d01 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:18.180854631 +0000 UTC m=+32.720299448 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-x67tb" (UID: "624e4d57-0db4-4be3-b512-e5065d800d01") : secret "networking-console-plugin-cert" not found
Nov 02 22:05:17.180972 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.180806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:17.180972 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.180896    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:17.180972 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.180909    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56dc494c89-qz46b: secret "image-registry-tls" not found
Nov 02 22:05:17.181197 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.180999    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls podName:1e6866c0-b229-4cd8-b977-32d366cced05 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:18.180987142 +0000 UTC m=+32.720431937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls") pod "image-registry-56dc494c89-qz46b" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05") : secret "image-registry-tls" not found
Nov 02 22:05:17.181197 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.181006    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:17.181197 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.181017    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b78b7d88-kdfn8: secret "image-registry-tls" not found
Nov 02 22:05:17.181197 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.181053    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls podName:a8b9eeac-3e3f-4612-a696-2ede201d83ac nodeName:}" failed. No retries permitted until 2026-11-02 22:05:18.181036495 +0000 UTC m=+32.720481305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls") pod "image-registry-66b78b7d88-kdfn8" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac") : secret "image-registry-tls" not found
Nov 02 22:05:17.181197 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.181075    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:18.181064797 +0000 UTC m=+32.720509610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : configmap references non-existent config key: service-ca.crt
Nov 02 22:05:17.181197 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.181114    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 22:05:17.181197 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.181156    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls podName:35ec85a5-34bf-4cd2-b209-7744c6f51a20 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:18.181144162 +0000 UTC m=+32.720588978 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-xf2zt" (UID: "35ec85a5-34bf-4cd2-b209-7744c6f51a20") : secret "samples-operator-tls" not found
Nov 02 22:05:17.181197 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.181172    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 22:05:17.181625 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.181237    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:18.181223052 +0000 UTC m=+32.720667860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : secret "router-metrics-certs-default" not found
Nov 02 22:05:17.197529 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.196298    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"]
Nov 02 22:05:17.204774 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:17.204722    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f32e923_7f5f_4ba7_b44d_6441fc46f876.slice/crio-45dcb6c4104c1e9e88e6a6aadb9896c44b62108d27ecd7981f8ba1f2fc77959c WatchSource:0}: Error finding container 45dcb6c4104c1e9e88e6a6aadb9896c44b62108d27ecd7981f8ba1f2fc77959c: Status 404 returned error can't find the container with id 45dcb6c4104c1e9e88e6a6aadb9896c44b62108d27ecd7981f8ba1f2fc77959c
Nov 02 22:05:17.233289 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.233242    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-9tsq4"]
Nov 02 22:05:17.251537 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:17.251495    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3465159d_8024_48a2_8408_eb1d1db96a8a.slice/crio-053148024fda9060b7d2525d3db4224d6b41cc51b333ce47287574995ea45d1f WatchSource:0}: Error finding container 053148024fda9060b7d2525d3db4224d6b41cc51b333ce47287574995ea45d1f: Status 404 returned error can't find the container with id 053148024fda9060b7d2525d3db4224d6b41cc51b333ce47287574995ea45d1f
Nov 02 22:05:17.264475 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.263503    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"]
Nov 02 22:05:17.267233 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.267159    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9pqwd"]
Nov 02 22:05:17.270399 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.270328    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"]
Nov 02 22:05:17.274901 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.274806    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-tjhsr" event={"ID":"101f50b9-bfd1-405a-ade4-6e1cc683dfda","Type":"ContainerStarted","Data":"3903975a55aacf078d9a316e1fe89ad9a10f24a1608e6425a2fade2399c2d77e"}
Nov 02 22:05:17.276320 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.276253    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xmv5g" event={"ID":"34ca0f75-5a97-4496-a7ae-060723b2d625","Type":"ContainerStarted","Data":"06840d7e46923a329bc487595c4b2397af3eb9999df2376a55adb9c57bf44261"}
Nov 02 22:05:17.277593 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.277526    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-7vddh" event={"ID":"012f51ba-d2ba-4d6b-b90d-67fe89601119","Type":"ContainerStarted","Data":"412cf88dada49b4ac00e82c9b197b7ae38c33a4e5ef99f4ccac1d9ba6d078a3c"}
Nov 02 22:05:17.281921 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.281840    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:17.282606 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.281982    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 22:05:17.282606 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:17.282044    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls podName:bfaa3942-3844-475e-9d7a-95bf64101115 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:18.28202814 +0000 UTC m=+32.821472957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls") pod "dns-default-gd729" (UID: "bfaa3942-3844-475e-9d7a-95bf64101115") : secret "dns-default-metrics-tls" not found
Nov 02 22:05:17.285558 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.285534    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-9tsq4" event={"ID":"3465159d-8024-48a2-8408-eb1d1db96a8a","Type":"ContainerStarted","Data":"053148024fda9060b7d2525d3db4224d6b41cc51b333ce47287574995ea45d1f"}
Nov 02 22:05:17.290613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.290545    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw" event={"ID":"8f32e923-7f5f-4ba7-b44d-6441fc46f876","Type":"ContainerStarted","Data":"45dcb6c4104c1e9e88e6a6aadb9896c44b62108d27ecd7981f8ba1f2fc77959c"}
Nov 02 22:05:17.520460 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.520426    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw"]
Nov 02 22:05:17.521556 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:17.521530    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4t246"]
Nov 02 22:05:17.524740 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:17.524714    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c712bb5_ab02_4ae7_8d21_b2274448ccf3.slice/crio-d7e82d762f215758ba4194f34d25af76ffb202a656126937f240eed35bacc818 WatchSource:0}: Error finding container d7e82d762f215758ba4194f34d25af76ffb202a656126937f240eed35bacc818: Status 404 returned error can't find the container with id d7e82d762f215758ba4194f34d25af76ffb202a656126937f240eed35bacc818
Nov 02 22:05:17.525358 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:17.525334    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dc1992e_41ec_4bde_9211_47162c8fb3c2.slice/crio-c2051f70262418c24128c001b5a9caa35b92623d209b9d67a2809142e501f2ef WatchSource:0}: Error finding container c2051f70262418c24128c001b5a9caa35b92623d209b9d67a2809142e501f2ef: Status 404 returned error can't find the container with id c2051f70262418c24128c001b5a9caa35b92623d209b9d67a2809142e501f2ef
Nov 02 22:05:18.089391 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.089349    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:18.089536 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.089494    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:18.089598 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.089564    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls podName:8759d3be-8ddd-4719-808f-3611fcd7f887 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:20.089542936 +0000 UTC m=+34.628987731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-rjgqf" (UID: "8759d3be-8ddd-4719-808f-3611fcd7f887") : secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:18.190231 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.190199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:18.190386 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.190246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:18.190386 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190345    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 22:05:18.190485 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190407    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 22:05:18.190485 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190413    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:20.190399432 +0000 UTC m=+34.729844226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : secret "router-metrics-certs-default" not found
Nov 02 22:05:18.190485 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.190470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:18.190613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.190504    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:18.190613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.190526    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:18.190613 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190532    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:18.190613 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190544    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b78b7d88-kdfn8: secret "image-registry-tls" not found
Nov 02 22:05:18.190613 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190558    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert podName:624e4d57-0db4-4be3-b512-e5065d800d01 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:20.190536262 +0000 UTC m=+34.729981068 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-x67tb" (UID: "624e4d57-0db4-4be3-b512-e5065d800d01") : secret "networking-console-plugin-cert" not found
Nov 02 22:05:18.190613 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190575    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 22:05:18.190613 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190582    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls podName:a8b9eeac-3e3f-4612-a696-2ede201d83ac nodeName:}" failed. No retries permitted until 2026-11-02 22:05:20.190571662 +0000 UTC m=+34.730016456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls") pod "image-registry-66b78b7d88-kdfn8" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac") : secret "image-registry-tls" not found
Nov 02 22:05:18.190613 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190603    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls podName:35ec85a5-34bf-4cd2-b209-7744c6f51a20 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:20.190594373 +0000 UTC m=+34.730039211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-xf2zt" (UID: "35ec85a5-34bf-4cd2-b209-7744c6f51a20") : secret "samples-operator-tls" not found
Nov 02 22:05:18.190613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.190605    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:18.190880 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190624    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:20.190612815 +0000 UTC m=+34.730057626 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : configmap references non-existent config key: service-ca.crt
Nov 02 22:05:18.190880 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190661    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:18.190880 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190688    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56dc494c89-qz46b: secret "image-registry-tls" not found
Nov 02 22:05:18.190880 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.190722    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls podName:1e6866c0-b229-4cd8-b977-32d366cced05 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:20.190713114 +0000 UTC m=+34.730157908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls") pod "image-registry-56dc494c89-qz46b" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05") : secret "image-registry-tls" not found
Nov 02 22:05:18.291890 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.291854    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:18.292303 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.291984    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 22:05:18.292303 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:18.292042    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls podName:bfaa3942-3844-475e-9d7a-95bf64101115 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:20.292024114 +0000 UTC m=+34.831468912 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls") pod "dns-default-gd729" (UID: "bfaa3942-3844-475e-9d7a-95bf64101115") : secret "dns-default-metrics-tls" not found
Nov 02 22:05:18.294875 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.294845    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-tjhsr" event={"ID":"101f50b9-bfd1-405a-ade4-6e1cc683dfda","Type":"ContainerStarted","Data":"412e7a115debb9d689246aa733e42398cce048203b976506603718a86bf70526"}
Nov 02 22:05:18.296357 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.296190    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xmv5g" event={"ID":"34ca0f75-5a97-4496-a7ae-060723b2d625","Type":"ContainerStarted","Data":"1ecc13fa92c424de5067a91d9b94a11b32d4c7efa09720a4844cd2f3c0366614"}
Nov 02 22:05:18.301646 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.301614    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-7vddh" event={"ID":"012f51ba-d2ba-4d6b-b90d-67fe89601119","Type":"ContainerStarted","Data":"91631cfb939f7cd3755ebed5ea3c71d6633e61c47946131c5e2501a2d86b453f"}
Nov 02 22:05:18.302726 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.302701    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4t246" event={"ID":"2c712bb5-ab02-4ae7-8d21-b2274448ccf3","Type":"ContainerStarted","Data":"d7e82d762f215758ba4194f34d25af76ffb202a656126937f240eed35bacc818"}
Nov 02 22:05:18.303926 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.303895    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw" event={"ID":"2dc1992e-41ec-4bde-9211-47162c8fb3c2","Type":"ContainerStarted","Data":"c2051f70262418c24128c001b5a9caa35b92623d209b9d67a2809142e501f2ef"}
Nov 02 22:05:18.305034 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.305012    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7" event={"ID":"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5","Type":"ContainerStarted","Data":"f4979c2e778dc685e1ae3d472886f13a4c79b86184a96735fe2781956a802e58"}
Nov 02 22:05:18.306083 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.306062    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9pqwd" event={"ID":"f020dd86-76b6-4433-a902-632d15269b91","Type":"ContainerStarted","Data":"abf0324e3eae007af025734a09c26e598e755f266ee20321ff4b57e2de8d2604"}
Nov 02 22:05:18.307200 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.307180    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8" event={"ID":"daa20bd0-d6e1-4044-812a-37ef99a19dac","Type":"ContainerStarted","Data":"ec43a28910e18cf590ef40259bea3bc03c37344c97b7d525ef89ade6a7870794"}
Nov 02 22:05:18.568951 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.567431    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-tjhsr" podStartSLOduration=432.567416517 podStartE2EDuration="7m12.567416517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:58:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:18.45635872 +0000 UTC m=+32.995803538" watchObservedRunningTime="2026-11-02 22:05:18.567416517 +0000 UTC m=+33.106861333"
Nov 02 22:05:18.681558 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.680143    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-xmv5g" podStartSLOduration=432.680127168 podStartE2EDuration="7m12.680127168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:58:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:18.679793103 +0000 UTC m=+33.219237974" watchObservedRunningTime="2026-11-02 22:05:18.680127168 +0000 UTC m=+33.219571988"
Nov 02 22:05:18.681558 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:18.681255    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-7vddh" podStartSLOduration=432.681240484 podStartE2EDuration="7m12.681240484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:58:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:18.568834224 +0000 UTC m=+33.108279040" watchObservedRunningTime="2026-11-02 22:05:18.681240484 +0000 UTC m=+33.220685505"
Nov 02 22:05:19.002199 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:19.001085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:19.002199 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:19.001135    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:19.002199 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:19.001314    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 22:05:19.002199 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:19.001378    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs podName:56b1aa47-8106-416e-869d-bfc5c10ef795 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:51.001350396 +0000 UTC m=+65.540795202 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs") pod "network-metrics-daemon-69fj9" (UID: "56b1aa47-8106-416e-869d-bfc5c10ef795") : secret "metrics-daemon-secret" not found
Nov 02 22:05:19.017583 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:19.017549    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:19.103908 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:19.103228    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:19.107035 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:19.106983    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:19.206780 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:19.206322    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:19.211075 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:19.211051    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:19.640628 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:19.640549    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7j964"]
Nov 02 22:05:19.692572 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:19.692515    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3505867a_1caf_4cef_b570_841afd5f7d85.slice/crio-47151337485e1db485942821f3b6c9bb5f0601eef1bf9c98e0de389666f3b6bb WatchSource:0}: Error finding container 47151337485e1db485942821f3b6c9bb5f0601eef1bf9c98e0de389666f3b6bb: Status 404 returned error can't find the container with id 47151337485e1db485942821f3b6c9bb5f0601eef1bf9c98e0de389666f3b6bb
Nov 02 22:05:19.695680 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:19.694983    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-78snq"]
Nov 02 22:05:20.113965 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.113936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:20.114142 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.114121    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:20.114223 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.114194    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls podName:8759d3be-8ddd-4719-808f-3611fcd7f887 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:24.114175905 +0000 UTC m=+38.653620711 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-rjgqf" (UID: "8759d3be-8ddd-4719-808f-3611fcd7f887") : secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.216296    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.216386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.216438    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.216488    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.216549    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.216572    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.216819    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.216836    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56dc494c89-qz46b: secret "image-registry-tls" not found
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.216895    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls podName:1e6866c0-b229-4cd8-b977-32d366cced05 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:24.216875432 +0000 UTC m=+38.756320234 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls") pod "image-registry-56dc494c89-qz46b" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05") : secret "image-registry-tls" not found
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.216973    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.217020    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:24.217008883 +0000 UTC m=+38.756453873 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : secret "router-metrics-certs-default" not found
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.217078    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.217113    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert podName:624e4d57-0db4-4be3-b512-e5065d800d01 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:24.217102994 +0000 UTC m=+38.756547788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-x67tb" (UID: "624e4d57-0db4-4be3-b512-e5065d800d01") : secret "networking-console-plugin-cert" not found
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.217171    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.217181    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b78b7d88-kdfn8: secret "image-registry-tls" not found
Nov 02 22:05:20.217433 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.217210    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls podName:a8b9eeac-3e3f-4612-a696-2ede201d83ac nodeName:}" failed. No retries permitted until 2026-11-02 22:05:24.217200937 +0000 UTC m=+38.756645730 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls") pod "image-registry-66b78b7d88-kdfn8" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac") : secret "image-registry-tls" not found
Nov 02 22:05:20.218412 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.217289    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:24.217267787 +0000 UTC m=+38.756712584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : configmap references non-existent config key: service-ca.crt
Nov 02 22:05:20.218412 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.217357    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 22:05:20.218412 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.217410    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls podName:35ec85a5-34bf-4cd2-b209-7744c6f51a20 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:24.217399068 +0000 UTC m=+38.756843862 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-xf2zt" (UID: "35ec85a5-34bf-4cd2-b209-7744c6f51a20") : secret "samples-operator-tls" not found
Nov 02 22:05:20.318059 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.317986    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:20.318197 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.318154    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 22:05:20.318235 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:20.318214    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls podName:bfaa3942-3844-475e-9d7a-95bf64101115 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:24.318197234 +0000 UTC m=+38.857642045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls") pod "dns-default-gd729" (UID: "bfaa3942-3844-475e-9d7a-95bf64101115") : secret "dns-default-metrics-tls" not found
Nov 02 22:05:20.332733 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.332704    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-78snq" event={"ID":"e47f56ea-3a73-4ff0-965d-3a21319aef2e","Type":"ContainerStarted","Data":"9913067ce49d832c2eab03f452f3832b5a201298145f349838b4ed777c55def4"}
Nov 02 22:05:20.334798 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.334747    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7j964" event={"ID":"3505867a-1caf-4cef-b570-841afd5f7d85","Type":"ContainerStarted","Data":"47151337485e1db485942821f3b6c9bb5f0601eef1bf9c98e0de389666f3b6bb"}
Nov 02 22:05:20.340448 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.340421    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw" event={"ID":"2dc1992e-41ec-4bde-9211-47162c8fb3c2","Type":"ContainerStarted","Data":"2ddf6690102e7f8596ae050b131991e794e10bf813d2d35f6c9b0c1e36233c1f"}
Nov 02 22:05:20.452394 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:20.452325    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw" podStartSLOduration=393.91496631 podStartE2EDuration="6m36.452308328s" totalImagesPullingTime="2.537342018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:20.451769387 +0000 UTC m=+34.991214204" watchObservedRunningTime="2026-11-02 22:05:20.452308328 +0000 UTC m=+34.991753192"
Nov 02 22:05:22.368433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:22.356502    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-9tsq4" event={"ID":"3465159d-8024-48a2-8408-eb1d1db96a8a","Type":"ContainerStarted","Data":"0b492fb520d1ba13609e8bc8f61b5bfdc4bfaf7f62f1a6db5d5b325d0d87b5d7"}
Nov 02 22:05:22.368433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:22.363066    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/0.log"
Nov 02 22:05:22.368433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:22.363104    2583 generic.go:356] "Generic (PLEG): container finished" podID="8f32e923-7f5f-4ba7-b44d-6441fc46f876" containerID="2627f551fcfbd6498fdd0a631c22c0c02160331e0ef529527b72215c184aede5" exitCode=255
Nov 02 22:05:22.368433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:22.363184    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw" event={"ID":"8f32e923-7f5f-4ba7-b44d-6441fc46f876","Type":"ContainerDied","Data":"2627f551fcfbd6498fdd0a631c22c0c02160331e0ef529527b72215c184aede5"}
Nov 02 22:05:22.368433 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:22.363524    2583 scope.go:122] "RemoveContainer" containerID="2627f551fcfbd6498fdd0a631c22c0c02160331e0ef529527b72215c184aede5"
Nov 02 22:05:22.480562 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:22.480116    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-9tsq4" podStartSLOduration=394.624071421 podStartE2EDuration="6m39.48009705s" totalImagesPullingTime="4.856025629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:22.478756644 +0000 UTC m=+37.018201460" watchObservedRunningTime="2026-11-02 22:05:22.48009705 +0000 UTC m=+37.019541866"
Nov 02 22:05:23.375092 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.374258    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7" event={"ID":"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5","Type":"ContainerStarted","Data":"2df6df52df6a30c7b856faebf62cc71d5d3d0a3253ae587870dc8a2389bfe913"}
Nov 02 22:05:23.378238 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.378122    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:05:23.379703 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.378642    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/0.log"
Nov 02 22:05:23.379703 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.378676    2583 generic.go:356] "Generic (PLEG): container finished" podID="8f32e923-7f5f-4ba7-b44d-6441fc46f876" containerID="8b54386b91d0c86d7932e0617cc2334f012249e97c574493e3d810f242944686" exitCode=255
Nov 02 22:05:23.379703 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.378748    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw" event={"ID":"8f32e923-7f5f-4ba7-b44d-6441fc46f876","Type":"ContainerDied","Data":"8b54386b91d0c86d7932e0617cc2334f012249e97c574493e3d810f242944686"}
Nov 02 22:05:23.379703 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.378796    2583 scope.go:122] "RemoveContainer" containerID="2627f551fcfbd6498fdd0a631c22c0c02160331e0ef529527b72215c184aede5"
Nov 02 22:05:23.379703 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.379186    2583 scope.go:122] "RemoveContainer" containerID="8b54386b91d0c86d7932e0617cc2334f012249e97c574493e3d810f242944686"
Nov 02 22:05:23.379703 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:23.379426    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-b5mdw_openshift-console-operator(8f32e923-7f5f-4ba7-b44d-6441fc46f876)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw" podUID="8f32e923-7f5f-4ba7-b44d-6441fc46f876"
Nov 02 22:05:23.384408 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.384320    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8" event={"ID":"daa20bd0-d6e1-4044-812a-37ef99a19dac","Type":"ContainerStarted","Data":"80e82ea7e935c62c94c0e03149cf851afc3da705eb840dd2b9fbe2abe5a5d233"}
Nov 02 22:05:23.487524 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.487471    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7" podStartSLOduration=397.199846015 podStartE2EDuration="6m42.48744915s" totalImagesPullingTime="5.287603135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:23.486585235 +0000 UTC m=+38.026030053" watchObservedRunningTime="2026-11-02 22:05:23.48744915 +0000 UTC m=+38.026893967"
Nov 02 22:05:23.712550 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.712439    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8" podStartSLOduration=403.200782618 podStartE2EDuration="6m48.712418512s" totalImagesPullingTime="5.511635894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:58:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:23.711937756 +0000 UTC m=+38.251382577" watchObservedRunningTime="2026-11-02 22:05:23.712418512 +0000 UTC m=+38.251863329"
Nov 02 22:05:23.869855 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.867794    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fdcvz"]
Nov 02 22:05:23.873378 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.870883    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:23.921301 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.921262    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 22:05:23.921498 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.921411    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 22:05:23.921562 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.921506    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q5sg6\""
Nov 02 22:05:23.927216 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.927186    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fdcvz"]
Nov 02 22:05:23.961058 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.961024    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:23.961241 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.961089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/985b7a7f-8711-4eec-8732-0fedb158f7a4-data-volume\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:23.961241 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.961202    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:23.961241 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.961236    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8b6z\" (UniqueName: \"kubernetes.io/projected/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-api-access-v8b6z\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:23.961461 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:23.961388    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/985b7a7f-8711-4eec-8732-0fedb158f7a4-crio-socket\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.062806 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.062768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.062974 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.062822    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8b6z\" (UniqueName: \"kubernetes.io/projected/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-api-access-v8b6z\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.062974 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.062919    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/985b7a7f-8711-4eec-8732-0fedb158f7a4-crio-socket\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.063070 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.062999    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.063070 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.063043    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/985b7a7f-8711-4eec-8732-0fedb158f7a4-data-volume\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.063431 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.063406    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/985b7a7f-8711-4eec-8732-0fedb158f7a4-data-volume\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.063544 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.063514    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/985b7a7f-8711-4eec-8732-0fedb158f7a4-crio-socket\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.063600 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.063588    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:24.063647 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.063638    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:24.563624207 +0000 UTC m=+39.103069013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:24.063763 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.063748    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 22:05:24.063844 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.063796    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:24.563781015 +0000 UTC m=+39.103225812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : configmap "kube-rbac-proxy" not found
Nov 02 22:05:24.123203 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.123155    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-qzwrq-r69mk"]
Nov 02 22:05:24.125902 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.125868    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8b6z\" (UniqueName: \"kubernetes.io/projected/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-api-access-v8b6z\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.127503 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.127466    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.163582 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.163544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:24.163964 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.163860    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:24.163964 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.163926    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls podName:8759d3be-8ddd-4719-808f-3611fcd7f887 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.163908337 +0000 UTC m=+46.703353135 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-rjgqf" (UID: "8759d3be-8ddd-4719-808f-3611fcd7f887") : secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:24.181558 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.181522    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-qzwrq-r69mk"]
Nov 02 22:05:24.265502 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.265464    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:24.265661 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.265534    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-archives-path\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.265661 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.265587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-trusted-ca-bundle\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.265736 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.265712    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:24.265772 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.265640    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 22:05:24.265811 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.265801    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.265785179 +0000 UTC m=+46.805229972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : secret "router-metrics-certs-default" not found
Nov 02 22:05:24.265849 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.265821    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 22:05:24.265877 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.265860    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert podName:624e4d57-0db4-4be3-b512-e5065d800d01 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.265848329 +0000 UTC m=+46.805293136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-x67tb" (UID: "624e4d57-0db4-4be3-b512-e5065d800d01") : secret "networking-console-plugin-cert" not found
Nov 02 22:05:24.265915 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.265743    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-service-ca-bundle\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.265915 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.265905    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:24.265975 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.265960    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:24.266006 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.265982    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:24.266118 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.266045    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:24.266118 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.266056    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b78b7d88-kdfn8: secret "image-registry-tls" not found
Nov 02 22:05:24.266266 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.266189    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 22:05:24.266336 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.266324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:24.266553 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.266458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b45cw\" (UniqueName: \"kubernetes.io/projected/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-kube-api-access-b45cw\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.266553 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.266470    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:24.266553 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.266486    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56dc494c89-qz46b: secret "image-registry-tls" not found
Nov 02 22:05:24.266553 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.266490    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls podName:a8b9eeac-3e3f-4612-a696-2ede201d83ac nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.266475813 +0000 UTC m=+46.805920622 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls") pod "image-registry-66b78b7d88-kdfn8" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac") : secret "image-registry-tls" not found
Nov 02 22:05:24.266553 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.266524    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.26650592 +0000 UTC m=+46.805950717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : configmap references non-existent config key: service-ca.crt
Nov 02 22:05:24.266553 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.266553    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls podName:35ec85a5-34bf-4cd2-b209-7744c6f51a20 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.266543278 +0000 UTC m=+46.805988122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-xf2zt" (UID: "35ec85a5-34bf-4cd2-b209-7744c6f51a20") : secret "samples-operator-tls" not found
Nov 02 22:05:24.266793 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.266575    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls podName:1e6866c0-b229-4cd8-b977-32d366cced05 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.266566728 +0000 UTC m=+46.806011526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls") pod "image-registry-56dc494c89-qz46b" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05") : secret "image-registry-tls" not found
Nov 02 22:05:24.370323 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.368010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-archives-path\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.370323 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.368056    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-trusted-ca-bundle\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.370323 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.368097    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:24.370323 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.368119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-service-ca-bundle\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.370323 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.368189    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b45cw\" (UniqueName: \"kubernetes.io/projected/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-kube-api-access-b45cw\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.370323 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.368862    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-archives-path\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.370323 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.369665    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-trusted-ca-bundle\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.370323 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.369759    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 22:05:24.370323 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.369807    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls podName:bfaa3942-3844-475e-9d7a-95bf64101115 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.369792009 +0000 UTC m=+46.909236818 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls") pod "dns-default-gd729" (UID: "bfaa3942-3844-475e-9d7a-95bf64101115") : secret "dns-default-metrics-tls" not found
Nov 02 22:05:24.370323 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.370238    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-service-ca-bundle\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.389909 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.389879    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:05:24.390314 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.390279    2583 scope.go:122] "RemoveContainer" containerID="8b54386b91d0c86d7932e0617cc2334f012249e97c574493e3d810f242944686"
Nov 02 22:05:24.390535 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.390509    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-b5mdw_openshift-console-operator(8f32e923-7f5f-4ba7-b44d-6441fc46f876)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw" podUID="8f32e923-7f5f-4ba7-b44d-6441fc46f876"
Nov 02 22:05:24.391686 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.391658    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7"]
Nov 02 22:05:24.394504 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.394477    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9pqwd" event={"ID":"f020dd86-76b6-4433-a902-632d15269b91","Type":"ContainerStarted","Data":"b6ddb65d8f3875af5fa4fbdf8cc70e0ed462be3bcecafaaecda862831cc4d8c4"}
Nov 02 22:05:24.394611 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.394567    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7"
Nov 02 22:05:24.406153 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.406133    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56"]
Nov 02 22:05:24.409041 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.409024    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56"
Nov 02 22:05:24.430807 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.430767    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b45cw\" (UniqueName: \"kubernetes.io/projected/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-kube-api-access-b45cw\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.445234 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.445208    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:24.445354 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.445274    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zz2sg\""
Nov 02 22:05:24.445354 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.445336    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 02 22:05:24.447622 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.447598    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:24.453955 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.453934    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7"]
Nov 02 22:05:24.466218 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.466196    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56"]
Nov 02 22:05:24.570613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.570560    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.570790 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.570707    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:24.570850 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.570838    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxf2x\" (UniqueName: \"kubernetes.io/projected/d987c59d-73b7-4883-b686-49809eeb776b-kube-api-access-lxf2x\") pod \"migrator-66df5d496b-m78f7\" (UID: \"d987c59d-73b7-4883-b686-49809eeb776b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7"
Nov 02 22:05:24.570929 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.570911    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x97rj\" (UniqueName: \"kubernetes.io/projected/1d714b69-b66b-46b4-8520-3537a5138ce3-kube-api-access-x97rj\") pod \"migrator-66df5d496b-ljl56\" (UID: \"1d714b69-b66b-46b4-8520-3537a5138ce3\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56"
Nov 02 22:05:24.574157 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.571590    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:24.574157 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.571654    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:25.571635749 +0000 UTC m=+40.111080557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:24.574157 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.573332    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 22:05:24.574157 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:24.573403    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:25.573385488 +0000 UTC m=+40.112830296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : configmap "kube-rbac-proxy" not found
Nov 02 22:05:24.671725 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.671696    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxf2x\" (UniqueName: \"kubernetes.io/projected/d987c59d-73b7-4883-b686-49809eeb776b-kube-api-access-lxf2x\") pod \"migrator-66df5d496b-m78f7\" (UID: \"d987c59d-73b7-4883-b686-49809eeb776b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7"
Nov 02 22:05:24.671850 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.671783    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x97rj\" (UniqueName: \"kubernetes.io/projected/1d714b69-b66b-46b4-8520-3537a5138ce3-kube-api-access-x97rj\") pod \"migrator-66df5d496b-ljl56\" (UID: \"1d714b69-b66b-46b4-8520-3537a5138ce3\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56"
Nov 02 22:05:24.734178 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.734145    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxf2x\" (UniqueName: \"kubernetes.io/projected/d987c59d-73b7-4883-b686-49809eeb776b-kube-api-access-lxf2x\") pod \"migrator-66df5d496b-m78f7\" (UID: \"d987c59d-73b7-4883-b686-49809eeb776b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7"
Nov 02 22:05:24.735489 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.735458    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x97rj\" (UniqueName: \"kubernetes.io/projected/1d714b69-b66b-46b4-8520-3537a5138ce3-kube-api-access-x97rj\") pod \"migrator-66df5d496b-ljl56\" (UID: \"1d714b69-b66b-46b4-8520-3537a5138ce3\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56"
Nov 02 22:05:24.743617 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.743562    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9pqwd" podStartSLOduration=2.5450414759999997 podStartE2EDuration="8.743543416s" totalImagesPullingTime="6.19850194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:24.742211755 +0000 UTC m=+39.281656572" watchObservedRunningTime="2026-11-02 22:05:24.743543416 +0000 UTC m=+39.282988235"
Nov 02 22:05:24.842991 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:24.842792    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-qzwrq-r69mk"]
Nov 02 22:05:24.846163 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:24.846132    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b23efe_ed57_42e2_bf7a_f51bb52e06c0.slice/crio-8a6946e110986354bec312abf97075b91e18efbcaf6a19244df0374b4834364c WatchSource:0}: Error finding container 8a6946e110986354bec312abf97075b91e18efbcaf6a19244df0374b4834364c: Status 404 returned error can't find the container with id 8a6946e110986354bec312abf97075b91e18efbcaf6a19244df0374b4834364c
Nov 02 22:05:25.009035 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.009004    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7"
Nov 02 22:05:25.021558 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.021526    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56"
Nov 02 22:05:25.385592 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.385558    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7"]
Nov 02 22:05:25.397945 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.397916    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-qzwrq-r69mk" event={"ID":"04b23efe-ed57-42e2-bf7a-f51bb52e06c0","Type":"ContainerStarted","Data":"200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b"}
Nov 02 22:05:25.398341 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.397979    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-qzwrq-r69mk" event={"ID":"04b23efe-ed57-42e2-bf7a-f51bb52e06c0","Type":"ContainerStarted","Data":"8a6946e110986354bec312abf97075b91e18efbcaf6a19244df0374b4834364c"}
Nov 02 22:05:25.399789 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.399765    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4t246" event={"ID":"2c712bb5-ab02-4ae7-8d21-b2274448ccf3","Type":"ContainerStarted","Data":"e5890b2d2a1a1888157eb017485b1123de2423308d274dc76683e70554183420"}
Nov 02 22:05:25.402099 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.402078    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7" event={"ID":"d987c59d-73b7-4883-b686-49809eeb776b","Type":"ContainerStarted","Data":"01d490a912e12dd21c1a1ed2130aa80483ce23fbdfd722b66d2f615243b14dd9"}
Nov 02 22:05:25.431078 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.431048    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56"]
Nov 02 22:05:25.442801 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:25.442313    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d714b69_b66b_46b4_8520_3537a5138ce3.slice/crio-80c1d4f4b772b1a8139116ccbdf9c3f10b05b78b4f280b838cfc72177bca4687 WatchSource:0}: Error finding container 80c1d4f4b772b1a8139116ccbdf9c3f10b05b78b4f280b838cfc72177bca4687: Status 404 returned error can't find the container with id 80c1d4f4b772b1a8139116ccbdf9c3f10b05b78b4f280b838cfc72177bca4687
Nov 02 22:05:25.511927 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.511518    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-qzwrq-r69mk" podStartSLOduration=1.5114983180000001 podStartE2EDuration="1.511498318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:25.51040625 +0000 UTC m=+40.049851067" watchObservedRunningTime="2026-11-02 22:05:25.511498318 +0000 UTC m=+40.050943133"
Nov 02 22:05:25.584611 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.584583    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:25.584754 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.584674    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:25.585323 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:25.585305    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:25.585404 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:25.585360    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:27.585346157 +0000 UTC m=+42.124790964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:25.585903 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:25.585887    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 22:05:25.585987 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:25.585935    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:27.585923618 +0000 UTC m=+42.125368425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : configmap "kube-rbac-proxy" not found
Nov 02 22:05:25.622325 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.622267    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4t246" podStartSLOduration=393.495581031 podStartE2EDuration="6m40.622246678s" totalImagesPullingTime="7.126665647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 21:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:25.621435708 +0000 UTC m=+40.160880525" watchObservedRunningTime="2026-11-02 22:05:25.622246678 +0000 UTC m=+40.161691495"
Nov 02 22:05:25.823779 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.823737    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-2fhhl"]
Nov 02 22:05:25.826313 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.826286    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:25.883235 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.883198    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 02 22:05:25.883476 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.883451    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 02 22:05:25.883476 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.883457    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 02 22:05:25.883613 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.883557    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bsff7\""
Nov 02 22:05:25.898998 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.898960    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-2fhhl"]
Nov 02 22:05:25.983797 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.983760    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 02 22:05:25.991922 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.991887    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-key\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:25.992114 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.992090    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-cabundle\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:25.992299 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:25.992278    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwsml\" (UniqueName: \"kubernetes.io/projected/2fe1c794-551a-40bf-b71a-4dc069aa0875-kube-api-access-qwsml\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:26.093576 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.093536    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-cabundle\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:26.093758 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.093724    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwsml\" (UniqueName: \"kubernetes.io/projected/2fe1c794-551a-40bf-b71a-4dc069aa0875-kube-api-access-qwsml\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:26.093758 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.093751    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-key\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:26.094164 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.094143    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-cabundle\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:26.096286 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.096262    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-key\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:26.153896 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.153855    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwsml\" (UniqueName: \"kubernetes.io/projected/2fe1c794-551a-40bf-b71a-4dc069aa0875-kube-api-access-qwsml\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:26.407640 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.407559    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7j964" event={"ID":"3505867a-1caf-4cef-b570-841afd5f7d85","Type":"ContainerStarted","Data":"b3f037dcf4e7a0762bbfcb82dc6786fb5d77723862a393dcede8a1ada0cc0080"}
Nov 02 22:05:26.408026 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.407715    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:26.409434 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.409392    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56" event={"ID":"1d714b69-b66b-46b4-8520-3537a5138ce3","Type":"ContainerStarted","Data":"80c1d4f4b772b1a8139116ccbdf9c3f10b05b78b4f280b838cfc72177bca4687"}
Nov 02 22:05:26.446098 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.446061    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:26.521326 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.521240    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7j964" podStartSLOduration=34.707127147 podStartE2EDuration="40.521226512s" totalImagesPullingTime="5.814099365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:26.520574089 +0000 UTC m=+41.060018904" watchObservedRunningTime="2026-11-02 22:05:26.521226512 +0000 UTC m=+41.060671327"
Nov 02 22:05:26.780727 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.780689    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:26.780913 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.780760    2583 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:26.781228 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.781209    2583 scope.go:122] "RemoveContainer" containerID="8b54386b91d0c86d7932e0617cc2334f012249e97c574493e3d810f242944686"
Nov 02 22:05:26.781494 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:26.781470    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-b5mdw_openshift-console-operator(8f32e923-7f5f-4ba7-b44d-6441fc46f876)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw" podUID="8f32e923-7f5f-4ba7-b44d-6441fc46f876"
Nov 02 22:05:26.826771 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:26.826736    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-2fhhl"]
Nov 02 22:05:26.834528 ip-10-0-130-41 kubenswrapper[2583]: W1102 22:05:26.834498    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe1c794_551a_40bf_b71a_4dc069aa0875.slice/crio-e38b1e2868e24f962fab0daae486edc1243ccc197c9ad59f73058dfafb61f360 WatchSource:0}: Error finding container e38b1e2868e24f962fab0daae486edc1243ccc197c9ad59f73058dfafb61f360: Status 404 returned error can't find the container with id e38b1e2868e24f962fab0daae486edc1243ccc197c9ad59f73058dfafb61f360
Nov 02 22:05:27.421794 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:27.416025    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl" event={"ID":"2fe1c794-551a-40bf-b71a-4dc069aa0875","Type":"ContainerStarted","Data":"757a82dce00b7b4d46d4a76b476e6a1465b61a3541f8c9c06d93a2f25bf7dea0"}
Nov 02 22:05:27.421794 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:27.416079    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl" event={"ID":"2fe1c794-551a-40bf-b71a-4dc069aa0875","Type":"ContainerStarted","Data":"e38b1e2868e24f962fab0daae486edc1243ccc197c9ad59f73058dfafb61f360"}
Nov 02 22:05:27.607918 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:27.607844    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:27.608057 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:27.607998    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 22:05:27.608100 ip-10-0-130-41 kubenswrapper[2583]: I1102 22:05:27.608049    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:27.608100 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:27.608078    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:31.608058368 +0000 UTC m=+46.147503176 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : configmap "kube-rbac-proxy" not found
Nov 02 22:05:27.608206 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:27.608136    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:27.608206 ip-10-0-130-41 kubenswrapper[2583]: E1102 22:05:27.608188    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:31.608175778 +0000 UTC m=+46.147620574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:27.654221 ip-10-0-130-41 systemd[1]: Stopping Kubernetes Kubelet...
Nov 02 22:05:27.669742 ip-10-0-130-41 systemd[1]: kubelet.service: Deactivated successfully.
Nov 02 22:05:27.669898 ip-10-0-130-41 systemd[1]: Stopped Kubernetes Kubelet.
Nov 02 22:05:27.670117 ip-10-0-130-41 systemd[1]: kubelet.service: Consumed 3.229s CPU time.
Nov 02 22:05:27.684958 ip-10-0-130-41 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 22:05:27.752866 ip-10-0-130-41 kubenswrapper[5067]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 22:05:27.752866 ip-10-0-130-41 kubenswrapper[5067]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 22:05:27.752866 ip-10-0-130-41 kubenswrapper[5067]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 22:05:27.752866 ip-10-0-130-41 kubenswrapper[5067]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 22:05:27.754915 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754833    5067 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:05:27.754915 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754914    5067 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:05:27.754915 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754918    5067 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754922    5067 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754925    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754928    5067 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754931    5067 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754934    5067 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754937    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754939    5067 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754942    5067 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754945    5067 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754947    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754950    5067 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754952    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754955    5067 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754957    5067 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754963    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754968    5067 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754972    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754976    5067 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:05:27.755026 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754979    5067 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754983    5067 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754985    5067 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754988    5067 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754991    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754994    5067 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754997    5067 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.754999    5067 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755002    5067 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755005    5067 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755008    5067 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755011    5067 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755013    5067 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755016    5067 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755018    5067 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755021    5067 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755024    5067 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755026    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755029    5067 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755033    5067 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:05:27.755621 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755035    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755038    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755041    5067 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755044    5067 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755046    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755049    5067 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755051    5067 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755054    5067 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755057    5067 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755060    5067 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755062    5067 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755065    5067 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755067    5067 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755070    5067 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755073    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755075    5067 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755077    5067 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755080    5067 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755083    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755085    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:05:27.756131 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755088    5067 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755090    5067 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755093    5067 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755095    5067 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755098    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755100    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755103    5067 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755106    5067 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755109    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755111    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755114    5067 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755116    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755119    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755121    5067 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755124    5067 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755129    5067 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755133    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755137    5067 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755140    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:05:27.756705 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755143    5067 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755146    5067 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755149    5067 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755151    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755154    5067 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755156    5067 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755159    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755162    5067 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755165    5067 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755169    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755173    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755177    5067 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755181    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755186    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755189    5067 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755193    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755199    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755202    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755205    5067 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:05:27.757188 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755207    5067 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755210    5067 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755212    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755215    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755218    5067 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755220    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.755223    5067 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756033    5067 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756040    5067 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756044    5067 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756047    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756049    5067 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756052    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756055    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756057    5067 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756061    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756064    5067 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756067    5067 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756069    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756074    5067 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:05:27.757666 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756078    5067 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756082    5067 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756086    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756089    5067 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756092    5067 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756095    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756098    5067 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756101    5067 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756104    5067 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756106    5067 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756109    5067 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756112    5067 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756115    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756117    5067 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756120    5067 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756122    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756125    5067 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756128    5067 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756130    5067 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:05:27.758159 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756133    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756137    5067 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756140    5067 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756143    5067 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756145    5067 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756148    5067 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756151    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756153    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756156    5067 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756159    5067 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756162    5067 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756165    5067 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756167    5067 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756170    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756173    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756175    5067 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756178    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756181    5067 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756184    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756187    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:05:27.758637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756189    5067 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756192    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756195    5067 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756197    5067 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756200    5067 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756202    5067 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756205    5067 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756208    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756211    5067 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756213    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756216    5067 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756219    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756221    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756224    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756227    5067 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756229    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756232    5067 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756235    5067 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756238    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756241    5067 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:05:27.759147 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756243    5067 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756246    5067 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756249    5067 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756252    5067 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756255    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756257    5067 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756260    5067 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756262    5067 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756265    5067 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756268    5067 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756270    5067 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756272    5067 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756275    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756277    5067 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756280    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756283    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756285    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756288    5067 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756291    5067 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:05:27.759662 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756293    5067 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756296    5067 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756298    5067 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756301    5067 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756303    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756322    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756325    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756330    5067 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756334    5067 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756337    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756340    5067 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756343    5067 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756345    5067 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756348    5067 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756351    5067 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756891    5067 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756897    5067 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756900    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756904    5067 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:05:27.760121 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756907    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756910    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756913    5067 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756916    5067 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756919    5067 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756921    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756924    5067 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756927    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756930    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756932    5067 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756935    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756938    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756941    5067 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756945    5067 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756949    5067 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756951    5067 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756954    5067 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756957    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756960    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:05:27.760670 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756963    5067 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756966    5067 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756969    5067 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756972    5067 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756982    5067 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756986    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756988    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756991    5067 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756994    5067 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.756997    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757000    5067 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757003    5067 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757005    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757008    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757011    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757013    5067 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757016    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757019    5067 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757021    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:05:27.761406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757024    5067 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757027    5067 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757030    5067 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757032    5067 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757035    5067 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757037    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757040    5067 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757045    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757047    5067 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757050    5067 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757053    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757055    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757060    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757062    5067 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757065    5067 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757068    5067 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757071    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757073    5067 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757076    5067 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:05:27.762241 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757078    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757081    5067 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757084    5067 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757086    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757089    5067 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757092    5067 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757095    5067 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757097    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757100    5067 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757102    5067 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757105    5067 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757107    5067 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757110    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757112    5067 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757115    5067 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757117    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757120    5067 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757123    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757125    5067 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757128    5067 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:05:27.763134 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757133    5067 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757137    5067 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757141    5067 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757143    5067 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757146    5067 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757150    5067 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757152    5067 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757155    5067 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757158    5067 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757161    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757163    5067 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757166    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757168    5067 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757171    5067 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757174    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757176    5067 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757178    5067 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757181    5067 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757185    5067 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757189    5067 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:05:27.764106 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757191    5067 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757194    5067 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757197    5067 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757200    5067 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757202    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757303    5067 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757313    5067 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757322    5067 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757330    5067 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757337    5067 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757340    5067 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757345    5067 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757350    5067 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757354    5067 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757357    5067 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757360    5067 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757382    5067 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757387    5067 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757390    5067 flags.go:64] FLAG: --cgroup-root=""
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757393    5067 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757396    5067 flags.go:64] FLAG: --client-ca-file=""
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757399    5067 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757402    5067 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757406    5067 flags.go:64] FLAG: --cluster-domain=""
Nov 02 22:05:27.765034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757410    5067 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757413    5067 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757416    5067 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757420    5067 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757424    5067 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757427    5067 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757430    5067 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757434    5067 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757437    5067 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757440    5067 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757443    5067 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757446    5067 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757449    5067 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757453    5067 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757456    5067 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757459    5067 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757462    5067 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757465    5067 flags.go:64] FLAG: --enable-server="true"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757468    5067 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757473    5067 flags.go:64] FLAG: --event-burst="100"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757477    5067 flags.go:64] FLAG: --event-qps="50"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757480    5067 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757484    5067 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757486    5067 flags.go:64] FLAG: --eviction-hard=""
Nov 02 22:05:27.766051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757491    5067 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757495    5067 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757498    5067 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757502    5067 flags.go:64] FLAG: --eviction-soft=""
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757505    5067 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757508    5067 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757511    5067 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757514    5067 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757517    5067 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757520    5067 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757523    5067 flags.go:64] FLAG: --feature-gates=""
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757527    5067 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757530    5067 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757534    5067 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757537    5067 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757540    5067 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757544    5067 flags.go:64] FLAG: --help="false"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757547    5067 flags.go:64] FLAG: --hostname-override="ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757551    5067 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757553    5067 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757556    5067 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757560    5067 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757563    5067 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 22:05:27.767121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757566    5067 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757569    5067 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757572    5067 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757575    5067 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757578    5067 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757582    5067 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757585    5067 flags.go:64] FLAG: --kube-reserved=""
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757588    5067 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757591    5067 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757594    5067 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757598    5067 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757602    5067 flags.go:64] FLAG: --lock-file=""
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757605    5067 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757608    5067 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757611    5067 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757616    5067 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757619    5067 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757622    5067 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757625    5067 flags.go:64] FLAG: --logging-format="text"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757628    5067 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757632    5067 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757635    5067 flags.go:64] FLAG: --manifest-url=""
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757638    5067 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757642    5067 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757645    5067 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 22:05:27.768193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757649    5067 flags.go:64] FLAG: --max-pods="110"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757652    5067 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757656    5067 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757659    5067 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757662    5067 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757665    5067 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757668    5067 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757671    5067 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757678    5067 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757693    5067 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757696    5067 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757699    5067 flags.go:64] FLAG: --pod-cidr=""
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757703    5067 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757707    5067 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757710    5067 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757712    5067 flags.go:64] FLAG: --port="10250"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757716    5067 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757720    5067 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-081e350f0ef2d6c35"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757723    5067 flags.go:64] FLAG: --qos-reserved=""
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757726    5067 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757729    5067 flags.go:64] FLAG: --register-node="true"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757732    5067 flags.go:64] FLAG: --register-with-taints=""
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757736    5067 flags.go:64] FLAG: --registry-burst="10"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757739    5067 flags.go:64] FLAG: --registry-qps="5"
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757742    5067 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757745    5067 flags.go:64] FLAG: --reserved-memory=""
Nov 02 22:05:27.769249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757749    5067 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757752    5067 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757755    5067 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757758    5067 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757761    5067 flags.go:64] FLAG: --runonce="false"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757764    5067 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757767    5067 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757770    5067 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757775    5067 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757778    5067 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757781    5067 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757784    5067 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757787    5067 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757790    5067 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757793    5067 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757796    5067 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757799    5067 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757802    5067 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757806    5067 flags.go:64] FLAG: --system-cgroups=""
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757809    5067 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757815    5067 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757818    5067 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757821    5067 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757826    5067 flags.go:64] FLAG: --tls-min-version=""
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757829    5067 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 22:05:27.770253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757833    5067 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757835    5067 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757838    5067 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757842    5067 flags.go:64] FLAG: --v="2"
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757846    5067 flags.go:64] FLAG: --version="false"
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757850    5067 flags.go:64] FLAG: --vmodule=""
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757854    5067 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.757858    5067 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757993    5067 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.757997    5067 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758000    5067 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758003    5067 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758006    5067 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758009    5067 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758012    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758014    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758018    5067 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758021    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758024    5067 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758026    5067 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758029    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:05:27.771307 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758032    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758034    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758037    5067 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758040    5067 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758044    5067 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758047    5067 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758049    5067 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758052    5067 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758055    5067 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758058    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758061    5067 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758064    5067 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758066    5067 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758069    5067 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758071    5067 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758074    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758077    5067 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758079    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758082    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758084    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:05:27.771936 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758087    5067 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758089    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758092    5067 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758095    5067 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758097    5067 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758100    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758102    5067 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758105    5067 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758108    5067 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758111    5067 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758114    5067 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758116    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758119    5067 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758121    5067 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758124    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758127    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758130    5067 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758133    5067 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758135    5067 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758138    5067 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:05:27.772589 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758141    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758145    5067 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758148    5067 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758150    5067 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758154    5067 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758158    5067 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758161    5067 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758163    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758166    5067 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758169    5067 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758172    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758174    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758177    5067 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758180    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758182    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758185    5067 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758189    5067 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758193    5067 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758197    5067 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:05:27.773194 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758201    5067 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758204    5067 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758207    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758211    5067 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758213    5067 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758216    5067 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758219    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758221    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758224    5067 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758227    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758230    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758232    5067 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758235    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758238    5067 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758241    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758244    5067 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758246    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758249    5067 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758251    5067 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:05:27.774216 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758254    5067 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758256    5067 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758259    5067 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758261    5067 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758264    5067 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758266    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758269    5067 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758272    5067 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758274    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758277    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758279    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758282    5067 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758285    5067 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758287    5067 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758290    5067 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.758316    5067 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 22:05:27.774770 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758373    5067 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758382    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758386    5067 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758388    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758391    5067 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758394    5067 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758396    5067 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758399    5067 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758401    5067 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758404    5067 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758407    5067 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758410    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758412    5067 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758415    5067 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758417    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758420    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758422    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758425    5067 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758428    5067 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:05:27.775230 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758431    5067 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758433    5067 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758436    5067 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758438    5067 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758441    5067 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758443    5067 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758446    5067 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758448    5067 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758451    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758453    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758456    5067 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758459    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758461    5067 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758464    5067 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758468    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758470    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758473    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758475    5067 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758478    5067 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758481    5067 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:05:27.775829 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758483    5067 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758486    5067 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758489    5067 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758491    5067 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758494    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758496    5067 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758499    5067 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758501    5067 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758504    5067 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758506    5067 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758509    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758512    5067 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758514    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758517    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758519    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758522    5067 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758524    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758527    5067 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758529    5067 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758532    5067 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:05:27.776477 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758534    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758537    5067 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758539    5067 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758542    5067 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758544    5067 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758547    5067 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758550    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758553    5067 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758555    5067 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758558    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758561    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758564    5067 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758566    5067 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758570    5067 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758573    5067 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758576    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758578    5067 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758581    5067 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758583    5067 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758587    5067 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:05:27.776970 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758591    5067 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758594    5067 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758597    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758600    5067 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758602    5067 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758605    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758607    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758610    5067 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758613    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758616    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758618    5067 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758622    5067 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758626    5067 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758629    5067 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758631    5067 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758634    5067 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758638    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758642    5067 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758645    5067 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:05:27.777478 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758647    5067 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758651    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758653    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758656    5067 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758659    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758662    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758665    5067 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.758669    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.763270    5067 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.763290    5067 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.763315    5067 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.763322    5067 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763425    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763434    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763439    5067 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763443    5067 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763448    5067 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763452    5067 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763457    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:05:27.777934 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763461    5067 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763465    5067 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763470    5067 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763474    5067 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763479    5067 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763483    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763488    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763492    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763496    5067 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763503    5067 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763509    5067 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763515    5067 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763521    5067 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763526    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763533    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763538    5067 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763544    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763549    5067 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763553    5067 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:05:27.778536 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763558    5067 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763563    5067 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763568    5067 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763572    5067 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763576    5067 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763580    5067 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763584    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763588    5067 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763592    5067 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763597    5067 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763601    5067 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763605    5067 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763609    5067 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763613    5067 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763617    5067 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763621    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763625    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763630    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763634    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763639    5067 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:05:27.779006 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763643    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763648    5067 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763652    5067 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763657    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763661    5067 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763665    5067 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763669    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763675    5067 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763683    5067 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763688    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763693    5067 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763697    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763702    5067 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763706    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763710    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763715    5067 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763719    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763723    5067 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763727    5067 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763731    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:05:27.779505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763736    5067 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763740    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763744    5067 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763749    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763753    5067 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763758    5067 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763762    5067 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763767    5067 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763771    5067 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763776    5067 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763780    5067 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763785    5067 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763789    5067 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763794    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763798    5067 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763802    5067 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763806    5067 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763811    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763815    5067 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:05:27.779983 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763820    5067 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763824    5067 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763830    5067 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763834    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763839    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763843    5067 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763848    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763852    5067 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763856    5067 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763860    5067 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763865    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763869    5067 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763873    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763877    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763882    5067 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763886    5067 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763890    5067 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763894    5067 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763898    5067 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763902    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:05:27.780520 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.763908    5067 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.763950    5067 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764173    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764182    5067 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764188    5067 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764193    5067 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764198    5067 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764202    5067 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764206    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764211    5067 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764215    5067 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764219    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764223    5067 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764227    5067 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764232    5067 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764237    5067 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:05:27.781009 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764242    5067 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764246    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764250    5067 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764255    5067 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764259    5067 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764263    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764267    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764271    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764275    5067 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764280    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764285    5067 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764291    5067 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764298    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764303    5067 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764307    5067 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764312    5067 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764316    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764320    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764325    5067 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:05:27.781426 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764329    5067 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764336    5067 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764341    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764345    5067 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764349    5067 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764354    5067 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764358    5067 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764378    5067 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764383    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764388    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764392    5067 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764397    5067 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764401    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764406    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764411    5067 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764415    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764420    5067 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764424    5067 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764428    5067 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:05:27.781871 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764433    5067 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764437    5067 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764442    5067 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764447    5067 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764451    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764456    5067 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764460    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764465    5067 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764469    5067 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764473    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764477    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764482    5067 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764492    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764496    5067 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764502    5067 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764506    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764511    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764515    5067 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764519    5067 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764524    5067 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:05:27.782443 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764528    5067 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764532    5067 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764536    5067 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764541    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764546    5067 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764550    5067 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764555    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764560    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764564    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764569    5067 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764573    5067 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764578    5067 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764582    5067 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764586    5067 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764592    5067 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764598    5067 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764603    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764607    5067 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764612    5067 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764616    5067 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:05:27.782935 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764620    5067 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764625    5067 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764629    5067 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764633    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764639    5067 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764644    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764651    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764655    5067 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764659    5067 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764664    5067 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764668    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764673    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764677    5067 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764681    5067 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.764722    5067 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764774    5067 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 22:05:27.783592 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764781    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764787    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764791    5067 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764796    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764801    5067 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764806    5067 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764811    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764815    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764820    5067 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764824    5067 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764828    5067 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764833    5067 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764837    5067 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764841    5067 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764845    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764850    5067 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764854    5067 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764859    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764863    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764867    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 22:05:27.784031 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764874    5067 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764878    5067 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764883    5067 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764888    5067 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764893    5067 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764899    5067 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764904    5067 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764910    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764915    5067 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764920    5067 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764924    5067 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764928    5067 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764933    5067 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764938    5067 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764942    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764947    5067 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764951    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764957    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764961    5067 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 22:05:27.784552 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764965    5067 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764970    5067 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764974    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764987    5067 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764991    5067 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.764995    5067 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765000    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765004    5067 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765009    5067 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765013    5067 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765017    5067 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765021    5067 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765026    5067 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765032    5067 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765036    5067 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765039    5067 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765045    5067 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765049    5067 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765053    5067 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765057    5067 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 22:05:27.785019 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765061    5067 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765065    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765070    5067 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765075    5067 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765079    5067 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765084    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765088    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765093    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765097    5067 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765102    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765106    5067 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765114    5067 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765121    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765126    5067 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765130    5067 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765134    5067 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765139    5067 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765143    5067 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765147    5067 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 22:05:27.785549 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765151    5067 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765156    5067 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765160    5067 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765164    5067 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765168    5067 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765173    5067 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765179    5067 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765183    5067 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765187    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765193    5067 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765197    5067 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765202    5067 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765206    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765210    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765214    5067 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765218    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765223    5067 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765227    5067 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765231    5067 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 22:05:27.786004 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765235    5067 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765239    5067 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765244    5067 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765249    5067 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765253    5067 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765257    5067 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765262    5067 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:27.765268    5067 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.765344    5067 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.766889    5067 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.766968    5067 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.767226    5067 server.go:1014] "Starting client certificate rotation"
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.767357    5067 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.767412    5067 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 21:59:46 +0000 UTC" deadline="2028-07-13 10:08:49.001960816 +0000 UTC"
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.767440    5067 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14844h3m21.234523009s"
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.767946    5067 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 22:05:27.786688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.770297    5067 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 22:05:27.787276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.772240    5067 log.go:25] "Validated CRI v1 runtime API"
Nov 02 22:05:27.787276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.773914    5067 log.go:25] "Validated CRI v1 image API"
Nov 02 22:05:27.787276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.774629    5067 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 22:05:27.787276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.776615    5067 fs.go:135] Filesystem UUIDs: map[3348d717-1d93-48bb-9667-14d0774f3849:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e74e0645-0c03-47ea-8b8d-a9684fff4664:/dev/nvme0n1p3]
Nov 02 22:05:27.787484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.776636    5067 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/01d490a912e12dd21c1a1ed2130aa80483ce23fbdfd722b66d2f615243b14dd9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/01d490a912e12dd21c1a1ed2130aa80483ce23fbdfd722b66d2f615243b14dd9/userdata/shm major:0 minor:342 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/053148024fda9060b7d2525d3db4224d6b41cc51b333ce47287574995ea45d1f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/053148024fda9060b7d2525d3db4224d6b41cc51b333ce47287574995ea45d1f/userdata/shm major:0 minor:233 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/06840d7e46923a329bc487595c4b2397af3eb9999df2376a55adb9c57bf44261/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/06840d7e46923a329bc487595c4b2397af3eb9999df2376a55adb9c57bf44261/userdata/shm major:0 minor:221 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0f1d9b7623f79f537e0583c362f333dead9b34cd227f2a6e7abec4c7d99b780e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0f1d9b7623f79f537e0583c362f333dead9b34cd227f2a6e7abec4c7d99b780e/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/21885d4b8115c0caba9a6b2aa5f7ab82895b1ca3d69ee5d45f6767938031df80/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/21885d4b8115c0caba9a6b2aa5f7ab82895b1ca3d69ee5d45f6767938031df80/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2505217cb809530edf93577485a6d4bc642d28d6b4e78eaac5490e713a1d207c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2505217cb809530edf93577485a6d4bc642d28d6b4e78eaac5490e713a1d207c/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3903975a55aacf078d9a316e1fe89ad9a10f24a1608e6425a2fade2399c2d77e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3903975a55aacf078d9a316e1fe89ad9a10f24a1608e6425a2fade2399c2d77e/userdata/shm major:0 minor:223 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/412cf88dada49b4ac00e82c9b197b7ae38c33a4e5ef99f4ccac1d9ba6d078a3c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/412cf88dada49b4ac00e82c9b197b7ae38c33a4e5ef99f4ccac1d9ba6d078a3c/userdata/shm major:0 minor:227 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/45dcb6c4104c1e9e88e6a6aadb9896c44b62108d27ecd7981f8ba1f2fc77959c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/45dcb6c4104c1e9e88e6a6aadb9896c44b62108d27ecd7981f8ba1f2fc77959c/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/47151337485e1db485942821f3b6c9bb5f0601eef1bf9c98e0de389666f3b6bb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/47151337485e1db485942821f3b6c9bb5f0601eef1bf9c98e0de389666f3b6bb/userdata/shm major:0 minor:288 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/52556c04a276b16fcc795dc1809d30cf1936abed59a4bb77d93375e921789a10/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/52556c04a276b16fcc795dc1809d30cf1936abed59a4bb77d93375e921789a10/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6031f68c17e487665f7a6e0c5ac48c2ec5b776e6585078c10c33f67c143311d8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6031f68c17e487665f7a6e0c5ac48c2ec5b776e6585078c10c33f67c143311d8/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7c4defbccef486e8f04eaf189fd4286e3d2f77ed2bedff1b3306e43abcc777f6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7c4defbccef486e8f04eaf189fd4286e3d2f77ed2bedff1b3306e43abcc777f6/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/80c1d4f4b772b1a8139116ccbdf9c3f10b05b78b4f280b838cfc72177bca4687/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/80c1d4f4b772b1a8139116ccbdf9c3f10b05b78b4f280b838cfc72177bca4687/userdata/shm major:0 minor:348 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8a042539c600ac0bfc6d488d141c75bfc471cfe6c9710d3aaad6a3a68c93b6ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8a042539c600ac0bfc6d488d141c75bfc471cfe6c9710d3aaad6a3a68c93b6ae/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8a6946e110986354bec312abf97075b91e18efbcaf6a19244df0374b4834364c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8a6946e110986354bec312abf97075b91e18efbcaf6a19244df0374b4834364c/userdata/shm major:0 minor:328 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8d2c2bf0dd57b393bea6095f331f626d58b06f89e8c1b15cb39afa7eefdcff4c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8d2c2bf0dd57b393bea6095f331f626d58b06f89e8c1b15cb39afa7eefdcff4c/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9913067ce49d832c2eab03f452f3832b5a201298145f349838b4ed777c55def4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9913067ce49d832c2eab03f452f3832b5a201298145f349838b4ed777c55def4/userdata/shm major:0 minor:289 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/abf0324e3eae007af025734a09c26e598e755f266ee20321ff4b57e2de8d2604/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/abf0324e3eae007af025734a09c26e598e755f266ee20321ff4b57e2de8d2604/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ae736b0b267bb64d8f407bf2cc055a3f7ebba2fc492253160e5be23f63236a0e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ae736b0b267bb64d8f407bf2cc055a3f7ebba2fc492253160e5be23f63236a0e/userdata/shm major:0 minor:89 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c2051f70262418c24128c001b5a9caa35b92623d209b9d67a2809142e501f2ef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c2051f70262418c24128c001b5a9caa35b92623d209b9d67a2809142e501f2ef/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cef0782836670b2f197f2f7b3dc587919b667417fa7d5bd22f36b9d04e71f51a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cef0782836670b2f197f2f7b3dc587919b667417fa7d5bd22f36b9d04e71f51a/userdata/shm major:0 minor:81 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d7e82d762f215758ba4194f34d25af76ffb202a656126937f240eed35bacc818/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d7e82d762f215758ba4194f34d25af76ffb202a656126937f240eed35bacc818/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e38b1e2868e24f962fab0daae486edc1243ccc197c9ad59f73058dfafb61f360/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e38b1e2868e24f962fab0daae486edc1243ccc197c9ad59f73058dfafb61f360/userdata/shm major:0 minor:362 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ec43a28910e18cf590ef40259bea3bc03c37344c97b7d525ef89ade6a7870794/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ec43a28910e18cf590ef40259bea3bc03c37344c97b7d525ef89ade6a7870794/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f4979c2e778dc685e1ae3d472886f13a4c79b86184a96735fe2781956a802e58/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f4979c2e778dc685e1ae3d472886f13a4c79b86184a96735fe2781956a802e58/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/012f51ba-d2ba-4d6b-b90d-67fe89601119/volumes/kubernetes.io~projected/kube-api-access-sfbdb:{mountpoint:/var/lib/kubelet/pods/012f51ba-d2ba-4d6b-b90d-67fe89601119/volumes/kubernetes.io~projected/kube-api-access-sfbdb major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/04b23efe-ed57-42e2-bf7a-f51bb52e06c0/volumes/kubernetes.io~projected/kube-api-access-b45cw:{mountpoint:/var/lib/kubelet/pods/04b23efe-ed57-42e2-bf7a-f51bb52e06c0/volumes/kubernetes.io~projected/kube-api-access-b45cw major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0caf91c1-266c-405f-92b4-070981db24c3/volumes/kubernetes.io~projected/kube-api-access-4r82z:{mountpoint:/var/lib/kubelet/pods/0caf91c1-266c-405f-92b4-070981db24c3/volumes/kubernetes.io~projected/kube-api-access-4r82z major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/101f50b9-bfd1-405a-ade4-6e1cc683dfda/volumes/kubernetes.io~projected/kube-api-access-8j2wx:{mountpoint:/var/lib/kubelet/pods/101f50b9-bfd1-405a-ade4-6e1cc683dfda/volumes/kubernetes.io~projected/kube-api-access-8j2wx major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1d714b69-b66b-46b4-8520-3537a5138ce3/volumes/kubernetes.io~projected/kube-api-access-x97rj:{mountpoint:/var/lib/kubelet/pods/1d714b69-b66b-46b4-8520-3537a5138ce3/volumes/kubernetes.io~projected/kube-api-access-x97rj major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~projected/kube-api-access-2dx64:{mountpoint:/var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~projected/kube-api-access-2dx64 major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e801619-9294-420b-8ad5-cd1005832f16/volumes/kubernetes.io~projected/kube-api-access-c94vr:{mountpoint:/var/lib/kubelet/pods/1e801619-9294-420b-8ad5-cd1005832f16/volumes/kubernetes.io~projected/kube-api-access-c94vr major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21f54242-5bab-4162-a435-85a080f8685c/volumes/kubernetes.io~projected/kube-api-access-jdx65:{mountpoint:/var/lib/kubelet/pods/21f54242-5bab-4162-a435-85a080f8685c/volumes/kubernetes.io~projected/kube-api-access-jdx65 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c712bb5-ab02-4ae7-8d21-b2274448ccf3/volumes/kubernetes.io~projected/kube-api-access-cqt4z:{mountpoint:/var/lib/kubelet/pods/2c712bb5-ab02-4ae7-8d21-b2274448ccf3/volumes/kubernetes.io~projected/kube-api-access-cqt4z major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2dc1992e-41ec-4bde-9211-47162c8fb3c2/volumes/kubernetes.io~projected/kube-api-access-t2t4v:{mountpoint:/var/lib/kubelet/pods/2dc1992e-41ec-4bde-9211-47162c8fb3c2/volumes/kubernetes.io~projected/kube-api-access-t2t4v major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2fe1c794-551a-40bf-b71a-4dc069aa0875/volumes/kubernetes.io~projected/kube-api-access-qwsml:{mountpoint:/var/lib/kubelet/pods/2fe1c794-551a-40bf-b71a-4dc069aa0875/volumes/kubernetes.io~projected/kube-api-access-qwsml major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2fe1c794-551a-40bf-b71a-4dc069aa0875/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/2fe1c794-551a-40bf-b71a-4dc069aa0875/volumes/kubernetes.io~secret/signing-key major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3465159d-8024-48a2-8408-eb1d1db96a8a/volumes/kubernetes.io~projected/kube-api-access-b66dw:{mountpoint:/var/lib/kubelet/pods/3465159d-8024-48a2-8408-eb1d1db96a8a/volumes/kubernetes.io~projected/kube-api-access-b66dw major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3465159d-8024-48a2-8408-eb1d1db96a8a/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/3465159d-8024-48a2-8408-eb1d1db96a8a/volumes/kubernetes.io~secret/serving-cert major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/34ca0f75-5a97-4496-a7ae-060723b2d625/volumes/kubernetes.io~projected/kube-api-access-s6zhx:{mountpoint:/var/lib/kubelet/pods/34ca0f75-5a97-4496-a7ae-060723b2d625/volumes/kubernetes.io~projected/kube-api-access-s6zhx major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3505867a-1caf-4cef-b570-841afd5f7d85/volumes/kubernetes.io~projected/kube-api-access-dmjvr:{mountpoint:/var/lib/kubelet/pods/3505867a-1caf-4cef-b570-841afd5f7d85/volumes/kubernetes.io~projected/kube-api-access-dmjvr major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/35ec85a5-34bf-4cd2-b209-7744c6f51a20/volumes/kubernetes.io~projected/kube-api-access-lfdlq:{mountpoint:/var/lib/kubelet/pods/35ec85a5-34bf-4cd2-b209-7744c6f51a20/volumes/kubernetes.io~projected/kube-api-access-lfdlq major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4843105e-92f4-4b83-83fd-06e7a217e009/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/4843105e-92f4-4b83-83fd-06e7a217e009/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/56b1aa47-8106-416e-869d-bfc5c10ef795/volumes/kubernetes.io~projected/kube-api-access-jtlnb:{mountpoint:/var/lib/kubelet/pods/56b1aa47-8106-416e-869d-bfc5c10ef795/volumes/kubernetes.io~projected/kube-api-access-jtlnb major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6b703071-5807-41b5-afe0-a2c85e23af47/volumes/kubernetes.io~projected/kube-api-access-d6f7s:{mountpoint:/var/lib/kubelet/pods/6b703071-5807-41b5-afe0-a2c85e23af47/volumes/kubernetes.io~projected/kube-api-access-d6f7s major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/713aba87-e440-4d2f-a339-52b3f222fa97/volumes/kubernetes.io~projected/kube-api-access-8dqqc:{mountpoint:/var/lib/kubelet/pods/713aba87-e440-4d2f-a339-52b3f222fa97/volumes/kubernetes.io~projected/kube-api-access-8dqqc major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8759d3be-8ddd-4719-808f-3611fcd7f887/volumes/kubernetes.io~projected/kube-api-access-zqjht:{mountpoint:/var/lib/kubelet/pods/8759d3be-8ddd-4719-808f-3611fcd7f887/volumes/kubernetes.io~projected/kube-api-access-zqjht major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f32e923-7f5f-4ba7-b44d-6441fc46f876/volumes/kubernetes.io~projected/kube-api-access-25h79:{mountpoint:/var/lib/kubelet/pods/8f32e923-7f5f-4ba7-b44d-6441fc46f876/volumes/kubernetes.io~projected/kube-api-access-25h79 major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f32e923-7f5f-4ba7-b44d-6441fc46f876/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/8f32e923-7f5f-4ba7-b44d-6441fc46f876/volumes/kubernetes.io~secret/serving-cert major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/985b7a7f-8711-4eec-8732-0fedb158f7a4/volumes/kubernetes.io~projected/kube-api-access-v8b6z:{mountpoint:/var/lib/kubelet/pods/985b7a7f-8711-4eec-8732-0fedb158f7a4/volumes/kubernetes.io~projected/kube-api-access-v8b6z major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9cfdf849-3285-4c0c-bcc8-2401667a460b/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/9cfdf849-3285-4c0c-bcc8-2401667a460b/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9cfdf849-3285-4c0c-bcc8-2401667a460b/volumes/kubernetes.io~projected/kube-api-access-tsh4k:{mountpoint:/var/lib/kubelet/pods/9cfdf849-3285-4c0c-bcc8-2401667a460b/volumes/kubernetes.io~projected/kube-api-access-tsh4k major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9cfdf849-3285-4c0c-bcc8-2401667a460b/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/9cfdf849-3285-4c0c-bcc8-2401667a460b/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5/volumes/kubernetes.io~projected/kube-api-access-stbfv:{mountpoint:/var/lib/kubelet/pods/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5/volumes/kubernetes.io~projected/kube-api-access-stbfv major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5/volumes/kubernetes.io~secret/serving-cert major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~projected/kube-api-access-7x47x:{mountpoint:/var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~projected/kube-api-access-7x47x major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bfaa3942-3844-475e-9d7a-95bf64101115/volumes/kubernetes.io~projected/kube-api-access-7ltdh:{mountpoint:/var/lib/kubelet/pods/bfaa3942-3844-475e-9d7a-95bf64101115/volumes/kubernetes.io~projected/kube-api-access-7ltdh major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d987c59d-73b7-4883-b686-49809eeb776b/volumes/kubernetes.io~projected/kube-api-access-lxf2x:{mountpoint:/var/lib/kubelet/pods/d987c59d-73b7-4883-b686-49809eeb776b/volumes/kubernetes.io~projected/kube-api-access-lxf2x major:0 minor:332 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/daa20bd0-d6e1-4044-812a-37ef99a19dac/volumes/kubernetes.io~projected/kube-api-access-q7wvw:{mountpoint:/var/lib/kubelet/pods/daa20bd0-d6e1-4044-812a-37ef99a19dac/volumes/kubernetes.io~projected/kube-api-access-q7wvw major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/daa20bd0-d6e1-4044-812a-37ef99a19dac/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/daa20bd0-d6e1-4044-812a-37ef99a19dac/volumes/kubernetes.io~secret/serving-cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dde46c9c-2501-4545-918c-9a3af3a6b381/volumes/kubernetes.io~projected/kube-api-access-w9fkg:{mountpoint:/var/lib/kubelet/pods/dde46c9c-2501-4545-918c-9a3af3a6b381/volumes/kubernetes.io~projected/kube-api-access-w9fkg major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dde46c9c-2501-4545-918c-9a3af3a6b381/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/dde46c9c-2501-4545-918c-9a3af3a6b381/volumes/kubernetes.io~secret/default-certificate major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dde46c9c-2501-4545-918c-9a3af3a6b381/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/dde46c9c-2501-4545-918c-9a3af3a6b381/volumes/kubernetes.io~secret/stats-auth major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4/volumes/kubernetes.io~projected/kube-api-access-5dq8x:{mountpoint:/var/lib/kubelet/pods/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4/volumes/kubernetes.io~projected/kube-api-access-5dq8x major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e47f56ea-3a73-4ff0-965d-3a21319aef2e/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/e47f56ea-3a73-4ff0-965d-3a21319aef2e/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f020dd86-76b6-4433-a902-632d15269b91/volumes/kubernetes.io~projected/kube-api-access-gls6g:{mountpoint:/var/lib/kubelet/pods/f020dd86-76b6-4433-a902-632d15269b91/volumes/kubernetes.io~projected/kube-api-access-gls6g major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f020dd86-76b6-4433-a902-632d15269b91/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/f020dd86-76b6-4433-a902-632d15269b91/volumes/kubernetes.io~secret/cert major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc25374f-cb3c-45ce-a523-af6fa11b570e/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/fc25374f-cb3c-45ce-a523-af6fa11b570e/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc25374f-cb3c-45ce-a523-af6fa11b570e/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/fc25374f-cb3c-45ce-a523-af6fa11b570e/volumes/kubernetes.io~empty-dir/tmp major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc25374f-cb3c-45ce-a523-af6fa11b570e/volumes/kubernetes.io~projected/kube-api-access-nff6m:{mountpoint:/var/lib/kubelet/pods/fc25374f-cb3c-45ce-a523-af6fa11b570e/volumes/kubernetes.io~projected/kube-api-access-nff6m major:0 minor:54 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/fff7d88f4b9989319d9fe320e69647e37f969056462934071954b478ab0152c0/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/8cd5797378a971aa9a4bd8cb59ab4ac2968a0ccb8f624136644122444aaeed1f/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/f7c1c60663b964bab9dc3cfeb538f9694446c1aefa9264057ee70f8e2d716175/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/6111ffd09b6640a836ccf9df6204b00866d2038ad56587a7e2e29a0a510acb63/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/750ef675d10d2a075699e3f4c6900919f5ee88ff9ef3798c6ff3ad0e053e1f6d/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/02ae837f629da7b7aca9f029c8a69fc572578ce2371044639c55756a830880cb/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/071a7b02e96994a86f2da970fd3920d11b2614d3c5bb8352e3ce3ec60da98e66/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/e999c19485de0ba6fbbdffe1ecf659aadd548e7d6cf3b006a989f72fddbf5794/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/87cc7f56041010b910d81887fb6ca3aadccaaa337c13595631f0358864404756/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/887e503bb9f0eeb33345bf164c29e962b7e9f5075ad96086122696e53b5ec1b9/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/76f8f6bf7e786130c1d73718488b857d493fe9a8b4ee2a6d5b389126dde58f51/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/65c5ba464ff152da608966997648ddf39d7d0b2b17760a038133d9c991461014/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/a061115c1ac149625530f87527121c1111d5a23873977c6acd228b81af659ef9/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/4ab48729c31a64faf3b3113291f948a9ef88d1c786a9aae6974575afe0c864e2/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/c37723c155ee7fb368380f209367131e9ea794fdffae89c4250363cba20d038d/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/a4308cdbfdb551afc167264a742ed53a95b86b7cc19fdd97f1ce62d17b040113/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/ef7b65db0637482baf2553256407175ec0ae47636226fae556a4b5840bf2488f/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/9b4791613aa3b51d57aed0aa0254fb4792ae66dcaa2ab79ff6c9e58be0b5f46d/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/94542d7a5cdd839c4c61c764bfb53621513b51b8b2627998b2c4dfb9212c48ca/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/bfa04da05027d8a904dfb4bb2d6af5419292b3cf0337a4b93426027a0a5e1177/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/7b15fc8a780eedc13a1795c67a5d477fa6994fce6cc82e9820cf8964877b6914/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/9ac90a842c58ddf7140c4dae77427618972913b353b9db78d4d8f75b3fb15fda/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/45c3f40a034eee5a32dcc28a9301158eb33d9f8edba4278f42231072301694e2/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/57c65197d194fa559139116059c6940accece8cefd3bee130fe4d33316bb29eb/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/7b4a347f4343b88903d1f89cb4b3440f1328ca289770be1fc0bc28d6d5d6a00d/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/c1392f63f4f03f2b967d8032593ff3f5d4a4b626e167e27adb880c03cc624447/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/8d4cd8c4bac0b7cc86355fdb9395a7d764c45033acef02649c3102c3143c9ad7/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/c86f7854ffa05cb8884415779f0323cc8f895e9192090aa7e9b645e4f9d02d83/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/cebb3d340b5c59311992c464c2b34c1f5582b7526062b4cc0d619b39e1aca9f4/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/0959f3eed5449e51d05943153dea2f21dc5f9e8aa0d312b6a7a4edaae837d861/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/1f1f1cea0b4bdb75219d2cc757e9f47312486c06594226ca9683cde311ea9242/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/cd7ae3a745d956e409df54963c59f37fac532c3fa1334363d6528ec3acc29f1a/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/207c75a11ce538e9a06b309ba5fe65e4d12a2e452b0fe595f2610c781944a17d/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/8cd2c380fe1e34fd8d2cf2f50cea127cc30d7e8f706198768f12abaf64b7ab11/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/031a819825a8af9642b6f4cbaca5dda272e0600e2dba41a1726e3092a74363e5/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/f0dadc60841cb28b48fe0cfe5f2f336c7605baf8abe255a8308b7800854d2d4a/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/4350047afbb7442f6d7ed97db55ee55043a25fa26c0783c78f15f9f9e879dd12/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/835e8edaaa642ab7a17b8261b0af087ffd21b1bd0c66123bca96437404f950ce/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/d535b1acb5035f8ac47c2851e5d6df70c08e37ca2771f2fdf5a2369a09a78837/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/2ebfea478b6b06694c3d4fc7a30a1b2e4f6175edc087edfb66815c230932eb6b/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/9bd51d43ce5c20425a1c61c1afbb2b1f2f603d968ffc94eadc61bf5da88ae605/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-330:{mountpoint:/var/lib/containers/storage/overlay/f6d400d6f0a53e87d6f6fe0c4a36ba7c83d96beb35cefb205212d3480c8fcb50/merged major:0 minor:330 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/c69d4c8ef7ebe3d0a5ddb8fedd2603f844d7d9498de7f360a60225c413d0e701/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-340:{mountpoint:/var/lib/containers/storage/overlay/68df3162e683530b8ef8eda141145fed168168ce689157a69640a1e7ce320d6c/merged major:0 minor:340 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/e08b2c548a4ec129225ec588a6a63c1f2345dcbbd0824f6138ca5af9270bbc99/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/85055617efbf49c9937827373c325316a85b57968510d1390c6f5db568e1cfa6/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/768aa4737bde88d562a65b912e9022a23ba2a3a7d285060b987ddb6daab2307a/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/d879f289d45d2bd87337eaac65ecb4659532e8075b99958b592d16b28e8d83d9/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/20016bb56ba986b4d2043c8be07d0efd005165779e09f5980751e9916c7c420f/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/2c3691938a8798113496117f987d6c985b4c7512c2e476a9245d865616f5a8f9/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-375:{mountpoint:/var/lib/containers/storage/overlay/f1414921b16ee1611ce222e2858fa09aaf62bdbe2471d8e8172572eaf94f8c74/merged major:0 minor:375 fsType:overlay blockSize:0} overlay_0-381:{mountpoint:/var/lib/containers/storage/overlay/19135f243e83015eef8da8b15d39d7930d3d463e7a774b41da45e24cfa974a17/merged major:0 minor:381 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/f35fd6a1acfbf7221804215ec8ad0ca3f9ea623e80924b7fcce3efee9d9a39f7/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/5fd169664183a3b9076f86face754c3fc1741c33505c786ac9390e5dfc454a96/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/ed598ba854f3f1edab3fa6062ec3b85c2d67a9034f6dca1cfa425de2f4e80ff9/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/5afcac43f6457dca296036463812f841b2c769cee7f82f06ffcde70ca29d8c7c/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/bdfdf9f28fc3b5b19614cdd1ea0b89d77303ed4621734669caaeb66013cbc745/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-83:{mountpoint:/var/lib/containers/storage/overlay/67d66286dd54da0ae0b983e62570a581258811f8eec325150874991ae0c9d170/merged major:0 minor:83 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/9faef972bea0de464ecaef81366991730bcba0e8c09d23951cbea112286a8d96/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-91:{mountpoint:/var/lib/containers/storage/overlay/ddb9882acf04a8ecb62294a434dfc30e87961a2d00ed8e442b2b6ada3888909f/merged major:0 minor:91 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/ccbe4576a35fd93e537ccc4ec99483dba361ee2b010dd25078ac7720d4447b24/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/c2b0de99a1bd3c21f543fe713963e31df04dbb3a79a83196396c47ec57a5338f/merged major:0 minor:98 fsType:overlay blockSize:0}]
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.784685    5067 manager.go:217] Machine: {Timestamp:2026-11-02 22:05:27.784199897 +0000 UTC m=+0.068009576 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101753 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2550167a686dc53f373d2f1190a696 SystemUUID:ec255016-7a68-6dc5-3f37-3d2f1190a696 BootID:d2c37894-583c-4ffb-853b-173f88d954db Filesystems:[{Device:/var/lib/kubelet/pods/8f32e923-7f5f-4ba7-b44d-6441fc46f876/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8a6946e110986354bec312abf97075b91e18efbcaf6a19244df0374b4834364c/userdata/shm DeviceMajor:0 DeviceMinor:328 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/daa20bd0-d6e1-4044-812a-37ef99a19dac/volumes/kubernetes.io~projected/kube-api-access-q7wvw DeviceMajor:0 DeviceMinor:215 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:207 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1d714b69-b66b-46b4-8520-3537a5138ce3/volumes/kubernetes.io~projected/kube-api-access-x97rj DeviceMajor:0 DeviceMinor:337 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0caf91c1-266c-405f-92b4-070981db24c3/volumes/kubernetes.io~projected/kube-api-access-4r82z DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7c4defbccef486e8f04eaf189fd4286e3d2f77ed2bedff1b3306e43abcc777f6/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bfaa3942-3844-475e-9d7a-95bf64101115/volumes/kubernetes.io~projected/kube-api-access-7ltdh DeviceMajor:0 DeviceMinor:237 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-340 DeviceMajor:0 DeviceMinor:340 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:212 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f020dd86-76b6-4433-a902-632d15269b91/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:226 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/412cf88dada49b4ac00e82c9b197b7ae38c33a4e5ef99f4ccac1d9ba6d078a3c/userdata/shm DeviceMajor:0 DeviceMinor:227 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9913067ce49d832c2eab03f452f3832b5a201298145f349838b4ed777c55def4/userdata/shm DeviceMajor:0 DeviceMinor:289 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2fe1c794-551a-40bf-b71a-4dc069aa0875/volumes/kubernetes.io~projected/kube-api-access-qwsml DeviceMajor:0 DeviceMinor:361 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0f1d9b7623f79f537e0583c362f333dead9b34cd227f2a6e7abec4c7d99b780e/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4843105e-92f4-4b83-83fd-06e7a217e009/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fc25374f-cb3c-45ce-a523-af6fa11b570e/volumes/kubernetes.io~projected/kube-api-access-nff6m DeviceMajor:0 DeviceMinor:54 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8a042539c600ac0bfc6d488d141c75bfc471cfe6c9710d3aaad6a3a68c93b6ae/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/52556c04a276b16fcc795dc1809d30cf1936abed59a4bb77d93375e921789a10/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/daa20bd0-d6e1-4044-812a-37ef99a19dac/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8f32e923-7f5f-4ba7-b44d-6441fc46f876/volumes/kubernetes.io~projected/kube-api-access-25h79 DeviceMajor:0 DeviceMinor:211 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3465159d-8024-48a2-8408-eb1d1db96a8a/volumes/kubernetes.io~projected/kube-api-access-b66dw DeviceMajor:0 DeviceMinor:213 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/06840d7e46923a329bc487595c4b2397af3eb9999df2376a55adb9c57bf44261/userdata/shm DeviceMajor:0 DeviceMinor:221 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2dc1992e-41ec-4bde-9211-47162c8fb3c2/volumes/kubernetes.io~projected/kube-api-access-t2t4v DeviceMajor:0 DeviceMinor:232 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/053148024fda9060b7d2525d3db4224d6b41cc51b333ce47287574995ea45d1f/userdata/shm DeviceMajor:0 DeviceMinor:233 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2c712bb5-ab02-4ae7-8d21-b2274448ccf3/volumes/kubernetes.io~projected/kube-api-access-cqt4z DeviceMajor:0 DeviceMinor:235 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cef0782836670b2f197f2f7b3dc587919b667417fa7d5bd22f36b9d04e71f51a/userdata/shm DeviceMajor:0 DeviceMinor:81 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~projected/kube-api-access-7x47x DeviceMajor:0 DeviceMinor:216 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~projected/kube-api-access-2dx64 DeviceMajor:0 DeviceMinor:218 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2fe1c794-551a-40bf-b71a-4dc069aa0875/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:356 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/35ec85a5-34bf-4cd2-b209-7744c6f51a20/volumes/kubernetes.io~projected/kube-api-access-lfdlq DeviceMajor:0 DeviceMinor:220 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9cfdf849-3285-4c0c-bcc8-2401667a460b/volumes/kubernetes.io~projected/kube-api-access-tsh4k DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:210 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/45dcb6c4104c1e9e88e6a6aadb9896c44b62108d27ecd7981f8ba1f2fc77959c/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f4979c2e778dc685e1ae3d472886f13a4c79b86184a96735fe2781956a802e58/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/01d490a912e12dd21c1a1ed2130aa80483ce23fbdfd722b66d2f615243b14dd9/userdata/shm DeviceMajor:0 DeviceMinor:342 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/04b23efe-ed57-42e2-bf7a-f51bb52e06c0/volumes/kubernetes.io~projected/kube-api-access-b45cw DeviceMajor:0 DeviceMinor:327 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-330 DeviceMajor:0 DeviceMinor:330 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3465159d-8024-48a2-8408-eb1d1db96a8a/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:208 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e38b1e2868e24f962fab0daae486edc1243ccc197c9ad59f73058dfafb61f360/userdata/shm DeviceMajor:0 DeviceMinor:362 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-381 DeviceMajor:0 DeviceMinor:381 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/21f54242-5bab-4162-a435-85a080f8685c/volumes/kubernetes.io~projected/kube-api-access-jdx65 DeviceMajor:0 DeviceMinor:51 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6031f68c17e487665f7a6e0c5ac48c2ec5b776e6585078c10c33f67c143311d8/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5/volumes/kubernetes.io~projected/kube-api-access-stbfv DeviceMajor:0 DeviceMinor:234 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/34ca0f75-5a97-4496-a7ae-060723b2d625/volumes/kubernetes.io~projected/kube-api-access-s6zhx DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dde46c9c-2501-4545-918c-9a3af3a6b381/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:205 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d7e82d762f215758ba4194f34d25af76ffb202a656126937f240eed35bacc818/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6b703071-5807-41b5-afe0-a2c85e23af47/volumes/kubernetes.io~projected/kube-api-access-d6f7s DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8759d3be-8ddd-4719-808f-3611fcd7f887/volumes/kubernetes.io~projected/kube-api-access-zqjht DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d987c59d-73b7-4883-b686-49809eeb776b/volumes/kubernetes.io~projected/kube-api-access-lxf2x DeviceMajor:0 DeviceMinor:332 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4/volumes/kubernetes.io~projected/kube-api-access-5dq8x DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dde46c9c-2501-4545-918c-9a3af3a6b381/volumes/kubernetes.io~projected/kube-api-access-w9fkg DeviceMajor:0 DeviceMinor:219 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:225 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8d2c2bf0dd57b393bea6095f331f626d58b06f89e8c1b15cb39afa7eefdcff4c/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3903975a55aacf078d9a316e1fe89ad9a10f24a1608e6425a2fade2399c2d77e/userdata/shm DeviceMajor:0 DeviceMinor:223 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3505867a-1caf-4cef-b570-841afd5f7d85/volumes/kubernetes.io~projected/kube-api-access-dmjvr DeviceMajor:0 DeviceMinor:287 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/47151337485e1db485942821f3b6c9bb5f0601eef1bf9c98e0de389666f3b6bb/userdata/shm DeviceMajor:0 DeviceMinor:288 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f020dd86-76b6-4433-a902-632d15269b91/volumes/kubernetes.io~projected/kube-api-access-gls6g DeviceMajor:0 DeviceMinor:231 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/56b1aa47-8106-416e-869d-bfc5c10ef795/volumes/kubernetes.io~projected/kube-api-access-jtlnb DeviceMajor:0 DeviceMinor:52 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c2051f70262418c24128c001b5a9caa35b92623d209b9d67a2809142e501f2ef/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9cfdf849-3285-4c0c-bcc8-2401667a460b/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:217 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1e801619-9294-420b-8ad5-cd1005832f16/volumes/kubernetes.io~projected/kube-api-access-c94vr DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/21885d4b8115c0caba9a6b2aa5f7ab82895b1ca3d69ee5d45f6767938031df80/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ae736b0b267bb64d8f407bf2cc055a3f7ebba2fc492253160e5be23f63236a0e/userdata/shm DeviceMajor:0 DeviceMinor:89 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/101f50b9-bfd1-405a-ade4-6e1cc683dfda/volumes/kubernetes.io~projected/kube-api-access-8j2wx DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dde46c9c-2501-4545-918c-9a3af3a6b381/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:209 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/abf0324e3eae007af025734a09c26e598e755f266ee20321ff4b57e2de8d2604/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fc25374f-cb3c-45ce-a523-af6fa11b570e/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:47 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/713aba87-e440-4d2f-a339-52b3f222fa97/volumes/kubernetes.io~projected/kube-api-access-8dqqc DeviceMajor:0 DeviceMinor:53 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9cfdf849-3285-4c0c-bcc8-2401667a460b/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:49 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2505217cb809530edf93577485a6d4bc642d28d6b4e78eaac5490e713a1d207c/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-91 DeviceMajor:0 DeviceMinor:91 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ec43a28910e18cf590ef40259bea3bc03c37344c97b7d525ef89ade6a7870794/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-83 DeviceMajor:0 DeviceMinor:83 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e47f56ea-3a73-4ff0-965d-3a21319aef2e/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:286 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/985b7a7f-8711-4eec-8732-0fedb158f7a4/volumes/kubernetes.io~projected/kube-api-access-v8b6z DeviceMajor:0 DeviceMinor:322 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fc25374f-cb3c-45ce-a523-af6fa11b570e/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:48 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/012f51ba-d2ba-4d6b-b90d-67fe89601119/volumes/kubernetes.io~projected/kube-api-access-sfbdb DeviceMajor:0 DeviceMinor:214 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/80c1d4f4b772b1a8139116ccbdf9c3f10b05b78b4f280b838cfc72177bca4687/userdata/shm DeviceMajor:0 DeviceMinor:348 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-375 DeviceMajor:0 DeviceMinor:375 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:01d490a912e12dd MacAddress:36:4d:94:86:61:4f Speed:10000 Mtu:8901} {Name:053148024fda906 MacAddress:7a:0a:b2:4c:ce:b6 Speed:10000 Mtu:8901} {Name:06840d7e46923a3 MacAddress:82:6b:3d:28:04:81 Speed:10000 Mtu:8901} {Name:3903975a55aacf0 MacAddress:82:a9:f2:de:7b:2a Speed:10000 Mtu:8901} {Name:412cf88dada49b4 MacAddress:0e:b8:50:db:72:41 Speed:10000 Mtu:8901} {Name:45dcb6c4104c1e9 MacAddress:36:a9:a9:8d:fa:07 Speed:10000 Mtu:8901} {Name:47151337485e1db MacAddress:42:c5:a3:b3:e3:94 Speed:10000 Mtu:8901} {Name:80c1d4f4b772b1a MacAddress:da:e6:f6:2d:59:72 Speed:10000 Mtu:8901} {Name:8a6946e11098635 MacAddress:e2:27:3f:d9:d3:ad Speed:10000 Mtu:8901} {Name:9913067ce49d832 MacAddress:8e:ae:24:36:0a:59 Speed:10000 Mtu:8901} {Name:abf0324e3eae007 MacAddress:3a:42:15:de:4c:09 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:fe:ce:96:93 Speed:0 Mtu:9001} {Name:br-int MacAddress:2a:f1:00:0e:7a:de Speed:0 Mtu:8901} {Name:c2051f70262418c MacAddress:ba:14:39:d1:db:85 Speed:10000 Mtu:8901} {Name:d7e82d762f21575 MacAddress:2a:6f:4f:a5:cc:3a Speed:10000 Mtu:8901} {Name:e38b1e2868e24f9 MacAddress:da:e4:97:c6:2b:76 Speed:10000 Mtu:8901} {Name:ec43a28910e18cf MacAddress:32:65:68:3b:3b:d6 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:fe:ce:96:93 Speed:0 Mtu:9001} {Name:f4979c2e778dc68 MacAddress:fa:e4:45:e2:03:9a Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:52:e8:99:a8:cc:f9 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:aa:9f:11:83:7a:76 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785045    5067 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785089    5067 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785390    5067 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785410    5067 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-41.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785543    5067 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785551    5067 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785566    5067 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785579    5067 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785670    5067 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785818    5067 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785943    5067 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785984    5067 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.785993    5067 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.786007    5067 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.786016    5067 kubelet.go:405] "Adding apiserver pod source"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.786025    5067 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.786689    5067 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 22:05:27.787851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.786700    5067 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 22:05:27.788779 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.788761    5067 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 22:05:27.790910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.790897    5067 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 22:05:27.790910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.790910    5067 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 22:05:27.791014 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791004    5067 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 02 22:05:27.791193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791182    5067 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 22:05:27.791226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791211    5067 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 22:05:27.791315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791306    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 22:05:27.791347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791322    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 22:05:27.791347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791328    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 22:05:27.791347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791334    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 22:05:27.791347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791340    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 22:05:27.791347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791346    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 22:05:27.791502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791352    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 22:05:27.791502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791358    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 22:05:27.791502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791381    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 22:05:27.791502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791387    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 22:05:27.791502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791396    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 22:05:27.791502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791405    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 22:05:27.791502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791429    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 22:05:27.791502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.791435    5067 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 22:05:27.794904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.794891    5067 server.go:1263] "Started kubelet"
Nov 02 22:05:27.794992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.794961    5067 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 22:05:27.796242 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.795288    5067 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 22:05:27.796242 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.795383    5067 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 22:05:27.796242 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.795616    5067 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 22:05:27.795469 ip-10-0-130-41 systemd[1]: Started Kubernetes Kubelet.
Nov 02 22:05:27.797798 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.796740    5067 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 22:05:27.799585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.799139    5067 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 22:05:27.799585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.799157    5067 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 22:05:27.799585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.799214    5067 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 22:05:27.799585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.799231    5067 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 22:05:27.799585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.799215    5067 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 21:59:47 +0000 UTC" deadline="2028-08-16 12:25:57.589018542 +0000 UTC"
Nov 02 22:05:27.799585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.799253    5067 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15662h20m29.789769283s"
Nov 02 22:05:27.799585 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:27.799283    5067 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-41.us-west-2.compute.internal\" not found"
Nov 02 22:05:27.799585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.799327    5067 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 22:05:27.802102 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.801585    5067 factory.go:55] Registering systemd factory
Nov 02 22:05:27.802102 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.801607    5067 factory.go:223] Registration of the systemd container factory successfully
Nov 02 22:05:27.806216 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806179    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04b23efe-ed57-42e2-bf7a-f51bb52e06c0" volumeName="kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-trusted-ca-bundle" seLinuxMountContext=""
Nov 02 22:05:27.806216 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806215    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0caf91c1-266c-405f-92b4-070981db24c3" volumeName="kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-multus-daemon-config" seLinuxMountContext=""
Nov 02 22:05:27.806398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806232    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4843105e-92f4-4b83-83fd-06e7a217e009" volumeName="kubernetes.io/secret/4843105e-92f4-4b83-83fd-06e7a217e009-agent-certs" seLinuxMountContext=""
Nov 02 22:05:27.806398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806247    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8b9eeac-3e3f-4612-a696-2ede201d83ac" volumeName="kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-installation-pull-secrets" seLinuxMountContext=""
Nov 02 22:05:27.806398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806260    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d987c59d-73b7-4883-b686-49809eeb776b" volumeName="kubernetes.io/projected/d987c59d-73b7-4883-b686-49809eeb776b-kube-api-access-lxf2x" seLinuxMountContext=""
Nov 02 22:05:27.806398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806271    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0caf91c1-266c-405f-92b4-070981db24c3" volumeName="kubernetes.io/projected/0caf91c1-266c-405f-92b4-070981db24c3-kube-api-access-4r82z" seLinuxMountContext=""
Nov 02 22:05:27.806398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806282    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="101f50b9-bfd1-405a-ade4-6e1cc683dfda" volumeName="kubernetes.io/projected/101f50b9-bfd1-405a-ade4-6e1cc683dfda-kube-api-access-8j2wx" seLinuxMountContext=""
Nov 02 22:05:27.806398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806293    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e6866c0-b229-4cd8-b977-32d366cced05" volumeName="kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-kube-api-access-2dx64" seLinuxMountContext=""
Nov 02 22:05:27.806398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806312    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c712bb5-ab02-4ae7-8d21-b2274448ccf3" volumeName="kubernetes.io/projected/2c712bb5-ab02-4ae7-8d21-b2274448ccf3-kube-api-access-cqt4z" seLinuxMountContext=""
Nov 02 22:05:27.806398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806340    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3465159d-8024-48a2-8408-eb1d1db96a8a" volumeName="kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-service-ca-bundle" seLinuxMountContext=""
Nov 02 22:05:27.806398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806378    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3465159d-8024-48a2-8408-eb1d1db96a8a" volumeName="kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-snapshots" seLinuxMountContext=""
Nov 02 22:05:27.806398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806391    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cfdf849-3285-4c0c-bcc8-2401667a460b" volumeName="kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-script-lib" seLinuxMountContext=""
Nov 02 22:05:27.806911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806402    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8b9eeac-3e3f-4612-a696-2ede201d83ac" volumeName="kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-certificates" seLinuxMountContext=""
Nov 02 22:05:27.806911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806433    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8b9eeac-3e3f-4612-a696-2ede201d83ac" volumeName="kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-kube-api-access-7x47x" seLinuxMountContext=""
Nov 02 22:05:27.806911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806477    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8b9eeac-3e3f-4612-a696-2ede201d83ac" volumeName="kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-bound-sa-token" seLinuxMountContext=""
Nov 02 22:05:27.806911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806491    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfaa3942-3844-475e-9d7a-95bf64101115" volumeName="kubernetes.io/projected/bfaa3942-3844-475e-9d7a-95bf64101115-kube-api-access-7ltdh" seLinuxMountContext=""
Nov 02 22:05:27.806911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806506    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" volumeName="kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 02 22:05:27.806911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806516    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" volumeName="kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-binary-copy" seLinuxMountContext=""
Nov 02 22:05:27.806911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806526    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc25374f-cb3c-45ce-a523-af6fa11b570e" volumeName="kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-tuned" seLinuxMountContext=""
Nov 02 22:05:27.806911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806533    5067 factory.go:153] Registering CRI-O factory
Nov 02 22:05:27.806911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806557    5067 factory.go:223] Registration of the crio container factory successfully
Nov 02 22:05:27.806911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806620    5067 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 22:05:27.807504 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.806539    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04b23efe-ed57-42e2-bf7a-f51bb52e06c0" volumeName="kubernetes.io/projected/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-kube-api-access-b45cw" seLinuxMountContext=""
Nov 02 22:05:27.807504 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807441    5067 factory.go:103] Registering Raw factory
Nov 02 22:05:27.807504 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807476    5067 manager.go:1199] Started watching for new ooms in manager
Nov 02 22:05:27.807693 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807471    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3465159d-8024-48a2-8408-eb1d1db96a8a" volumeName="kubernetes.io/projected/3465159d-8024-48a2-8408-eb1d1db96a8a-kube-api-access-b66dw" seLinuxMountContext=""
Nov 02 22:05:27.807791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807699    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2c3d84e-0830-4800-bb72-28bd2b6ca4c5" volumeName="kubernetes.io/secret/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-serving-cert" seLinuxMountContext=""
Nov 02 22:05:27.807791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807722    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="012f51ba-d2ba-4d6b-b90d-67fe89601119" volumeName="kubernetes.io/projected/012f51ba-d2ba-4d6b-b90d-67fe89601119-kube-api-access-sfbdb" seLinuxMountContext=""
Nov 02 22:05:27.807791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807745    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fe1c794-551a-40bf-b71a-4dc069aa0875" volumeName="kubernetes.io/configmap/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-cabundle" seLinuxMountContext=""
Nov 02 22:05:27.807791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807759    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cfdf849-3285-4c0c-bcc8-2401667a460b" volumeName="kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-config" seLinuxMountContext=""
Nov 02 22:05:27.807791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807772    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dde46c9c-2501-4545-918c-9a3af3a6b381" volumeName="kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-stats-auth" seLinuxMountContext=""
Nov 02 22:05:27.807946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807794    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="985b7a7f-8711-4eec-8732-0fedb158f7a4" volumeName="kubernetes.io/projected/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-api-access-v8b6z" seLinuxMountContext=""
Nov 02 22:05:27.807946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807808    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04b23efe-ed57-42e2-bf7a-f51bb52e06c0" volumeName="kubernetes.io/empty-dir/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-archives-path" seLinuxMountContext=""
Nov 02 22:05:27.807946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807836    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e801619-9294-420b-8ad5-cd1005832f16" volumeName="kubernetes.io/projected/1e801619-9294-420b-8ad5-cd1005832f16-kube-api-access-c94vr" seLinuxMountContext=""
Nov 02 22:05:27.807946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807864    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3505867a-1caf-4cef-b570-841afd5f7d85" volumeName="kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr" seLinuxMountContext=""
Nov 02 22:05:27.807946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807877    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b703071-5807-41b5-afe0-a2c85e23af47" volumeName="kubernetes.io/projected/6b703071-5807-41b5-afe0-a2c85e23af47-kube-api-access-d6f7s" seLinuxMountContext=""
Nov 02 22:05:27.807946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807892    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8b9eeac-3e3f-4612-a696-2ede201d83ac" volumeName="kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-trusted-ca" seLinuxMountContext=""
Nov 02 22:05:27.807946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807910    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8b9eeac-3e3f-4612-a696-2ede201d83ac" volumeName="kubernetes.io/empty-dir/a8b9eeac-3e3f-4612-a696-2ede201d83ac-ca-trust-extracted" seLinuxMountContext=""
Nov 02 22:05:27.807946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807944    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cfdf849-3285-4c0c-bcc8-2401667a460b" volumeName="kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-env-overrides" seLinuxMountContext=""
Nov 02 22:05:27.808175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807966    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e6866c0-b229-4cd8-b977-32d366cced05" volumeName="kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-registry-certificates" seLinuxMountContext=""
Nov 02 22:05:27.808175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807981    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e6866c0-b229-4cd8-b977-32d366cced05" volumeName="kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-trusted-ca" seLinuxMountContext=""
Nov 02 22:05:27.808175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807995    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e6866c0-b229-4cd8-b977-32d366cced05" volumeName="kubernetes.io/empty-dir/1e6866c0-b229-4cd8-b977-32d366cced05-ca-trust-extracted" seLinuxMountContext=""
Nov 02 22:05:27.808175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808015    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3465159d-8024-48a2-8408-eb1d1db96a8a" volumeName="kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-tmp" seLinuxMountContext=""
Nov 02 22:05:27.808175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.807950    5067 manager.go:319] Starting recovery of all containers
Nov 02 22:05:27.808315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808200    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cfdf849-3285-4c0c-bcc8-2401667a460b" volumeName="kubernetes.io/projected/9cfdf849-3285-4c0c-bcc8-2401667a460b-kube-api-access-tsh4k" seLinuxMountContext=""
Nov 02 22:05:27.808315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808220    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3465159d-8024-48a2-8408-eb1d1db96a8a" volumeName="kubernetes.io/secret/3465159d-8024-48a2-8408-eb1d1db96a8a-serving-cert" seLinuxMountContext=""
Nov 02 22:05:27.808315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808240    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4843105e-92f4-4b83-83fd-06e7a217e009" volumeName="kubernetes.io/configmap/4843105e-92f4-4b83-83fd-06e7a217e009-konnectivity-ca" seLinuxMountContext=""
Nov 02 22:05:27.808315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808254    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56b1aa47-8106-416e-869d-bfc5c10ef795" volumeName="kubernetes.io/projected/56b1aa47-8106-416e-869d-bfc5c10ef795-kube-api-access-jtlnb" seLinuxMountContext=""
Nov 02 22:05:27.808315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808268    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8759d3be-8ddd-4719-808f-3611fcd7f887" volumeName="kubernetes.io/configmap/8759d3be-8ddd-4719-808f-3611fcd7f887-telemetry-config" seLinuxMountContext=""
Nov 02 22:05:27.808315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808288    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" volumeName="kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 02 22:05:27.808315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808304    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f020dd86-76b6-4433-a902-632d15269b91" volumeName="kubernetes.io/projected/f020dd86-76b6-4433-a902-632d15269b91-kube-api-access-gls6g" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808324    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e6866c0-b229-4cd8-b977-32d366cced05" volumeName="kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-bound-sa-token" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808338    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="35ec85a5-34bf-4cd2-b209-7744c6f51a20" volumeName="kubernetes.io/configmap/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-config" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808354    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="713aba87-e440-4d2f-a339-52b3f222fa97" volumeName="kubernetes.io/configmap/713aba87-e440-4d2f-a339-52b3f222fa97-serviceca" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808385    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f32e923-7f5f-4ba7-b44d-6441fc46f876" volumeName="kubernetes.io/projected/8f32e923-7f5f-4ba7-b44d-6441fc46f876-kube-api-access-25h79" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808406    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2c3d84e-0830-4800-bb72-28bd2b6ca4c5" volumeName="kubernetes.io/projected/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-kube-api-access-stbfv" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808423    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dde46c9c-2501-4545-918c-9a3af3a6b381" volumeName="kubernetes.io/projected/dde46c9c-2501-4545-918c-9a3af3a6b381-kube-api-access-w9fkg" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808438    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0caf91c1-266c-405f-92b4-070981db24c3" volumeName="kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-cni-binary-copy" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808454    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="daa20bd0-d6e1-4044-812a-37ef99a19dac" volumeName="kubernetes.io/configmap/daa20bd0-d6e1-4044-812a-37ef99a19dac-config" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808474    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dde46c9c-2501-4545-918c-9a3af3a6b381" volumeName="kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-default-certificate" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808491    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f020dd86-76b6-4433-a902-632d15269b91" volumeName="kubernetes.io/secret/f020dd86-76b6-4433-a902-632d15269b91-cert" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808512    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21f54242-5bab-4162-a435-85a080f8685c" volumeName="kubernetes.io/projected/21f54242-5bab-4162-a435-85a080f8685c-kube-api-access-jdx65" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808556    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3465159d-8024-48a2-8408-eb1d1db96a8a" volumeName="kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-trusted-ca-bundle" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808572    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="34ca0f75-5a97-4496-a7ae-060723b2d625" volumeName="kubernetes.io/projected/34ca0f75-5a97-4496-a7ae-060723b2d625-kube-api-access-s6zhx" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808591    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f32e923-7f5f-4ba7-b44d-6441fc46f876" volumeName="kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-config" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808603    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="daa20bd0-d6e1-4044-812a-37ef99a19dac" volumeName="kubernetes.io/projected/daa20bd0-d6e1-4044-812a-37ef99a19dac-kube-api-access-q7wvw" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808617    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e47f56ea-3a73-4ff0-965d-3a21319aef2e" volumeName="kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808634    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc25374f-cb3c-45ce-a523-af6fa11b570e" volumeName="kubernetes.io/projected/fc25374f-cb3c-45ce-a523-af6fa11b570e-kube-api-access-nff6m" seLinuxMountContext=""
Nov 02 22:05:27.808621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808648    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1d714b69-b66b-46b4-8520-3537a5138ce3" volumeName="kubernetes.io/projected/1d714b69-b66b-46b4-8520-3537a5138ce3-kube-api-access-x97rj" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808665    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="35ec85a5-34bf-4cd2-b209-7744c6f51a20" volumeName="kubernetes.io/projected/35ec85a5-34bf-4cd2-b209-7744c6f51a20-kube-api-access-lfdlq" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808678    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f32e923-7f5f-4ba7-b44d-6441fc46f876" volumeName="kubernetes.io/secret/8f32e923-7f5f-4ba7-b44d-6441fc46f876-serving-cert" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808694    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="daa20bd0-d6e1-4044-812a-37ef99a19dac" volumeName="kubernetes.io/secret/daa20bd0-d6e1-4044-812a-37ef99a19dac-serving-cert" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808710    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" volumeName="kubernetes.io/projected/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-kube-api-access-5dq8x" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808722    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2dc1992e-41ec-4bde-9211-47162c8fb3c2" volumeName="kubernetes.io/projected/2dc1992e-41ec-4bde-9211-47162c8fb3c2-kube-api-access-t2t4v" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808740    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fe1c794-551a-40bf-b71a-4dc069aa0875" volumeName="kubernetes.io/projected/2fe1c794-551a-40bf-b71a-4dc069aa0875-kube-api-access-qwsml" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808753    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fe1c794-551a-40bf-b71a-4dc069aa0875" volumeName="kubernetes.io/secret/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-key" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808767    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="713aba87-e440-4d2f-a339-52b3f222fa97" volumeName="kubernetes.io/projected/713aba87-e440-4d2f-a339-52b3f222fa97-kube-api-access-8dqqc" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808783    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cfdf849-3285-4c0c-bcc8-2401667a460b" volumeName="kubernetes.io/secret/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808795    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfaa3942-3844-475e-9d7a-95bf64101115" volumeName="kubernetes.io/empty-dir/bfaa3942-3844-475e-9d7a-95bf64101115-tmp-dir" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808812    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e6866c0-b229-4cd8-b977-32d366cced05" volumeName="kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-installation-pull-secrets" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808828    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e801619-9294-420b-8ad5-cd1005832f16" volumeName="kubernetes.io/configmap/1e801619-9294-420b-8ad5-cd1005832f16-iptables-alerter-script" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808846    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21f54242-5bab-4162-a435-85a080f8685c" volumeName="kubernetes.io/empty-dir/21f54242-5bab-4162-a435-85a080f8685c-tmp-dir" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808858    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f32e923-7f5f-4ba7-b44d-6441fc46f876" volumeName="kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-trusted-ca" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808872    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2c3d84e-0830-4800-bb72-28bd2b6ca4c5" volumeName="kubernetes.io/configmap/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-config" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808889    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfaa3942-3844-475e-9d7a-95bf64101115" volumeName="kubernetes.io/configmap/bfaa3942-3844-475e-9d7a-95bf64101115-config-volume" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808905    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04b23efe-ed57-42e2-bf7a-f51bb52e06c0" volumeName="kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-service-ca-bundle" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808924    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e6866c0-b229-4cd8-b977-32d366cced05" volumeName="kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-image-registry-private-configuration" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808940    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="624e4d57-0db4-4be3-b512-e5065d800d01" volumeName="kubernetes.io/configmap/624e4d57-0db4-4be3-b512-e5065d800d01-nginx-conf" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808953    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8759d3be-8ddd-4719-808f-3611fcd7f887" volumeName="kubernetes.io/projected/8759d3be-8ddd-4719-808f-3611fcd7f887-kube-api-access-zqjht" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.808985    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="985b7a7f-8711-4eec-8732-0fedb158f7a4" volumeName="kubernetes.io/empty-dir/985b7a7f-8711-4eec-8732-0fedb158f7a4-data-volume" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.809000    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8b9eeac-3e3f-4612-a696-2ede201d83ac" volumeName="kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-image-registry-private-configuration" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.809018    5067 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc25374f-cb3c-45ce-a523-af6fa11b570e" volumeName="kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-tmp" seLinuxMountContext=""
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.809031    5067 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 22:05:27.809482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.809039    5067 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 22:05:27.821189 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.821150    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:05:27.821897 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.821879    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:05:27.835880 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.835744    5067 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 22:05:27.837513 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.837492    5067 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 22:05:27.837605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.837522    5067 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 22:05:27.837605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.837543    5067 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 22:05:27.837605 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:27.837584    5067 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 02 22:05:27.848358 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.848337    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:05:27.848358 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.848380    5067 generic.go:356] "Generic (PLEG): container finished" podID="8f32e923-7f5f-4ba7-b44d-6441fc46f876" containerID="8b54386b91d0c86d7932e0617cc2334f012249e97c574493e3d810f242944686" exitCode=255
Nov 02 22:05:27.870425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.870342    5067 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="29ea596b952de5e75c9b14f35f5bd20a58dc54cb426621fb24778267ba59a97b" exitCode=0
Nov 02 22:05:27.870425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.870385    5067 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="c965d3d57831dadfa49959f7b95f850209de58db15bf2ed48495b67dedd319d2" exitCode=0
Nov 02 22:05:27.870425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.870394    5067 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="7b4e58ea47dd6d78818f1082c951403e8007a2dbf1e796e6e0776cba78bd8f18" exitCode=0
Nov 02 22:05:27.870425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.870400    5067 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="8367bd33fdd34ac4c521dc481bb88be969217b2b3437f96622740f8a9ba05378" exitCode=0
Nov 02 22:05:27.870425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.870408    5067 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="8abb0d692616ce8d49943ab3c4fb7ee09e6a9878ba90d6312167de10b0070bb9" exitCode=0
Nov 02 22:05:27.870425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.870416    5067 generic.go:356] "Generic (PLEG): container finished" podID="e2bc1cd9-8631-4e6e-82be-053f3d8df4d4" containerID="b64b7225e9c97e1d9796f652d6f3ee43af837b0a80ea6fc146fdb4845b734ff1" exitCode=0
Nov 02 22:05:27.886252 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.886218    5067 generic.go:356] "Generic (PLEG): container finished" podID="657ace375fa97dc1e56f2b6e64a6ac61" containerID="3adf846d77093c82ddc07986ec525030916968fcb2ccfa72c431fc621a816505" exitCode=0
Nov 02 22:05:27.899358 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:27.899332    5067 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-41.us-west-2.compute.internal\" not found"
Nov 02 22:05:27.912311 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.912289    5067 manager.go:324] Recovery completed
Nov 02 22:05:27.937778 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:27.937734    5067 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 02 22:05:27.940200 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.940177    5067 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 22:05:27.941595 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.941575    5067 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 22:05:27.941595 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.941609    5067 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 22:05:27.941780 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.941622    5067 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 22:05:27.944250 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944233    5067 cpu_manager.go:226] "Starting" policy="none"
Nov 02 22:05:27.944250 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944245    5067 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 22:05:27.944441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944265    5067 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 22:05:27.944441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944246    5067 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 22:05:27.944441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944422    5067 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 02 22:05:27.944600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944433    5067 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 02 22:05:27.944600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944451    5067 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 02 22:05:27.944600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944456    5067 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 02 22:05:27.944600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944462    5067 policy_none.go:50] "Start"
Nov 02 22:05:27.944600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944471    5067 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 22:05:27.944600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944481    5067 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 22:05:27.944870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944606    5067 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 02 22:05:27.944870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944613    5067 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 02 22:05:27.944870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.944628    5067 policy_none.go:44] "Start"
Nov 02 22:05:27.945312 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.945296    5067 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 22:05:27.945418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.945327    5067 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 22:05:27.945418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.945341    5067 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 22:05:27.949222 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.949190    5067 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 22:05:27.952708 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.952692    5067 manager.go:341] "Starting Device Plugin manager"
Nov 02 22:05:27.952790 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:27.952718    5067 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 22:05:27.952790 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.952727    5067 server.go:87] "Starting device plugin registration server"
Nov 02 22:05:27.953057 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.953040    5067 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 22:05:27.953138 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.953057    5067 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 22:05:27.953189 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.953166    5067 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 22:05:27.953576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.953537    5067 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 22:05:27.953675 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.953586    5067 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 22:05:27.953675 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.953593    5067 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 22:05:27.953773 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.953652    5067 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T22:05:27.953573223Z","UUID":"34ecf191-a879-4ec9-a35e-2f567dba054c","Handler":null,"Name":"","Endpoint":""}
Nov 02 22:05:27.959514 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:27.954886    5067 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 22:05:27.959514 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:27.954926    5067 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-41.us-west-2.compute.internal\" not found"
Nov 02 22:05:27.959514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.955920    5067 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 22:05:27.959514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.955952    5067 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 22:05:27.987126 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.987095    5067 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 22:05:27.987126 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.987127    5067 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 22:05:27.987352 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.987228    5067 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 02 22:05:27.987730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:27.987683    5067 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 22:05:28.033821 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.033790    5067 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 22:05:28.054045 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.054018    5067 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 22:05:28.055396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.055354    5067 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 02 22:05:28.055509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.055408    5067 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 22:05:28.055509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.055419    5067 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-41.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 02 22:05:28.055509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.055464    5067 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.138111 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.138019    5067 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"]
Nov 02 22:05:28.138807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.138767    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal" event={"ID":"376e7bd75c236d364d4749b5f447d5f4","Type":"ContainerStarted","Data":"4eea97239249f65ac4c30eea9101948bec459506cac63d3694472b9ce5cc40c9"}
Nov 02 22:05:28.138946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.138812    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal" event={"ID":"376e7bd75c236d364d4749b5f447d5f4","Type":"ContainerStarted","Data":"8a042539c600ac0bfc6d488d141c75bfc471cfe6c9710d3aaad6a3a68c93b6ae"}
Nov 02 22:05:28.138946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.138853    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal" event={"ID":"657ace375fa97dc1e56f2b6e64a6ac61","Type":"ContainerStarted","Data":"3ab2fbc96ec4a60b5ab9020c9cae0340c55e96d81120f5ab6e5f61042f525819"}
Nov 02 22:05:28.138946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.138782    5067 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.139152 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.138871    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal" event={"ID":"657ace375fa97dc1e56f2b6e64a6ac61","Type":"ContainerDied","Data":"3adf846d77093c82ddc07986ec525030916968fcb2ccfa72c431fc621a816505"}
Nov 02 22:05:28.139152 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.138987    5067 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.139152 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.138984    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal" event={"ID":"657ace375fa97dc1e56f2b6e64a6ac61","Type":"ContainerStarted","Data":"7c4defbccef486e8f04eaf189fd4286e3d2f77ed2bedff1b3306e43abcc777f6"}
Nov 02 22:05:28.158481 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.158458    5067 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.158565 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.158513    5067 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.198335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.198308    5067 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 22:05:28.198502 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:28.198431    5067 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.198731 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.198717    5067 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 22:05:28.198785 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:28.198749    5067 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.290803 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.289098    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/376e7bd75c236d364d4749b5f447d5f4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"376e7bd75c236d364d4749b5f447d5f4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.290803 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.289189    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.290803 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.289228    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.390701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.390421    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.390701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.390459    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.390701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.390523    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.390701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.390551    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/376e7bd75c236d364d4749b5f447d5f4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"376e7bd75c236d364d4749b5f447d5f4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.390701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.390606    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/376e7bd75c236d364d4749b5f447d5f4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"376e7bd75c236d364d4749b5f447d5f4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.390701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.390639    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/657ace375fa97dc1e56f2b6e64a6ac61-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" (UID: \"657ace375fa97dc1e56f2b6e64a6ac61\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.786407 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.786356    5067 apiserver.go:52] "Watching apiserver"
Nov 02 22:05:28.887553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.887527    5067 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 22:05:28.888210 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.888183    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-v76fm","openshift-insights/periodic-gathering-qzwrq-r69mk","openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7","kube-system/kas-connection-checker-685895c7c9-tjhsr","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw","openshift-network-diagnostics/network-check-target-7j964","kube-system/kas-connection-checker-685895c7c9-xmv5g","openshift-ingress-canary/ingress-canary-9pqwd","kube-system/konnectivity-agent-ww4zl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr","openshift-insights/insights-operator-7c4986d449-9tsq4","openshift-multus/multus-dkbhl","openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8","openshift-image-registry/image-registry-66b78b7d88-kdfn8","openshift-cluster-node-tuning-operator/tuned-f4n2z","openshift-dns/dns-default-gd729","openshift-image-registry/image-registry-56dc494c89-qz46b","openshift-dns/node-resolver-t5mqk","openshift-network-diagnostics/network-check-source-6f78d59f99-4t246","openshift-network-console/networking-console-plugin-86c478bb88-x67tb","openshift-console-operator/console-operator-7cd56b8dbc-b5mdw","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt","openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7","openshift-insights/insights-runtime-extractor-fdcvz","openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56","openshift-service-ca/service-ca-5fffc97bdf-2fhhl","kube-system/kas-connection-checker-685895c7c9-7vddh","openshift-multus/network-metrics-daemon-69fj9","kube-system/global-pull-secret-syncer-78snq","openshift-multus/multus-additional-cni-plugins-xbz67","openshift-network-operator/iptables-alerter-bpxkm","kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal","openshift-ovn-kubernetes/ovnkube-node-5jljx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal","openshift-ingress/router-default-b448bfb57-lttxh"]
Nov 02 22:05:28.890068 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.890044    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:28.890414 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.890397    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:28.891305 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.891288    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:28.892320 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.892303    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.896499 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.896472    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:28.899740 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.899718    5067 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.899815 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.899790    5067 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.901913 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.901892    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:28.903931 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.903912    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:28.904471 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.904446    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:28.905841 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.905823    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:28.938348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.938317    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.938533 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.938519    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.938682 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.938669    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 02 22:05:28.938749 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.938703    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 02 22:05:28.938810 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.938791    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 02 22:05:28.939302 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.939284    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 02 22:05:28.939423 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.939411    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.939534 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.939515    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 02 22:05:28.939582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.939548    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.939661 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.939621    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 02 22:05:28.940337 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.940315    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 02 22:05:28.940531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.940449    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jtvd2\""
Nov 02 22:05:28.940585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.940537    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 02 22:05:28.940693 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.940638    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-89mts\""
Nov 02 22:05:28.940813 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.940756    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.941229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.941207    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.941609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.941591    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 02 22:05:28.941830 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.941810    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.942107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.942091    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.942225 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.942192    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.942572 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.942548    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 02 22:05:28.943699 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.943680    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 02 22:05:28.943797 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.943726    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 02 22:05:28.943876 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.943861    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 02 22:05:28.943927 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.943872    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.943977 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.943942    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.943977 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.943948    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.983825 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.983796    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.986204 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986182    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.986419 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986397    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 22:05:28.986494 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986468    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.986549 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986523    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.986601 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986565    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.986601 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986572    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.986601 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986565    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 22:05:28.986754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986602    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.986754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986612    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.986754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986612    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.986754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986640    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 22:05:28.986754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986659    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.986754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986665    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.986754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986671    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.986754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986699    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.986754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986710    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986822    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986840    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986848    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986853    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986884    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986901    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986916    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986922    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986926    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986947    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986945    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986961    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.986976    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.987003    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.987135    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 22:05:28.987194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.987189    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 22:05:28.987663 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.987253    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 22:05:28.987663 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.987315    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.990066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990048    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 22:05:28.990145 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990067    5067 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 22:05:28.990201 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:28.990154    5067 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.990265 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990250    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.990331 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990271    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.990331 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990290    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 02 22:05:28.990331 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990303    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 02 22:05:28.990483 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990334    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.990483 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990357    5067 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 22:05:28.990483 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990434    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.990483 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990437    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 02 22:05:28.990682 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:28.990542    5067 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-41.us-west-2.compute.internal"
Nov 02 22:05:28.990682 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.990621    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 22:05:28.991992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.991972    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 02 22:05:28.991992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.991975    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 02 22:05:28.993627 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993609    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e801619-9294-420b-8ad5-cd1005832f16-iptables-alerter-script\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:05:28.993695 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993637    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e801619-9294-420b-8ad5-cd1005832f16-host-slash\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:05:28.993695 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993653    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:28.993695 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993670    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9fkg\" (UniqueName: \"kubernetes.io/projected/dde46c9c-2501-4545-918c-9a3af3a6b381-kube-api-access-w9fkg\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:28.993695 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993692    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-config\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.993868 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993851    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-tmp\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:28.993908 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993882    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/713aba87-e440-4d2f-a339-52b3f222fa97-host\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:05:28.993908 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993904    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dqqc\" (UniqueName: \"kubernetes.io/projected/713aba87-e440-4d2f-a339-52b3f222fa97-kube-api-access-8dqqc\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:05:28.993988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993927    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c94vr\" (UniqueName: \"kubernetes.io/projected/1e801619-9294-420b-8ad5-cd1005832f16-kube-api-access-c94vr\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:05:28.993988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993949    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j2wx\" (UniqueName: \"kubernetes.io/projected/101f50b9-bfd1-405a-ade4-6e1cc683dfda-kube-api-access-8j2wx\") pod \"kas-connection-checker-685895c7c9-tjhsr\" (UID: \"101f50b9-bfd1-405a-ade4-6e1cc683dfda\") " pod="kube-system/kas-connection-checker-685895c7c9-tjhsr"
Nov 02 22:05:28.993988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993971    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsh4k\" (UniqueName: \"kubernetes.io/projected/9cfdf849-3285-4c0c-bcc8-2401667a460b-kube-api-access-tsh4k\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.994134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.993998    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfdlq\" (UniqueName: \"kubernetes.io/projected/35ec85a5-34bf-4cd2-b209-7744c6f51a20-kube-api-access-lfdlq\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:28.994134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994025    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:28.994134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994065    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-tmp\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:28.994134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994082    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e801619-9294-420b-8ad5-cd1005832f16-iptables-alerter-script\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:05:28.994134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994100    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-config\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.994288 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994070    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-bound-sa-token\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.994288 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994164    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-slash\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.994288 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994193    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-env-overrides\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.994288 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994221    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3465159d-8024-48a2-8408-eb1d1db96a8a-serving-cert\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:28.994440 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994316    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8b9eeac-3e3f-4612-a696-2ede201d83ac-ca-trust-extracted\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.994440 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994407    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8b9eeac-3e3f-4612-a696-2ede201d83ac-ca-trust-extracted\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.994510 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994437    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/daa20bd0-d6e1-4044-812a-37ef99a19dac-serving-cert\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:28.994510 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994475    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-env-overrides\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.994578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994545    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-etc-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.994578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994566    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-config\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:28.994702 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994584    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:28.994702 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994601    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stbfv\" (UniqueName: \"kubernetes.io/projected/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-kube-api-access-stbfv\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:28.994806 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994761    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-trusted-ca\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.994806 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994796    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfbdb\" (UniqueName: \"kubernetes.io/projected/012f51ba-d2ba-4d6b-b90d-67fe89601119-kube-api-access-sfbdb\") pod \"kas-connection-checker-685895c7c9-7vddh\" (UID: \"012f51ba-d2ba-4d6b-b90d-67fe89601119\") " pod="kube-system/kas-connection-checker-685895c7c9-7vddh"
Nov 02 22:05:28.994904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994825    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-systemd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.994904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994830    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-config\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:28.994904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994830    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/daa20bd0-d6e1-4044-812a-37ef99a19dac-serving-cert\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:28.994904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994854    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.994904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994855    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:28.994904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.994882    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.995119 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995016    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/713aba87-e440-4d2f-a339-52b3f222fa97-serviceca\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:05:28.995119 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995047    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-kubelet\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.995119 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995062    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-log-socket\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.995119 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995078    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-installation-pull-secrets\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.995119 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995114    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x47x\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-kube-api-access-7x47x\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.995301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995136    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-trusted-ca\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.995301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995157    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:28.995301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995261    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqjht\" (UniqueName: \"kubernetes.io/projected/8759d3be-8ddd-4719-808f-3611fcd7f887-kube-api-access-zqjht\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:28.995301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995292    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-stats-auth\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:28.995509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995354    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-systemd-units\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.995509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995403    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-node-log\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.995509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995432    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/713aba87-e440-4d2f-a339-52b3f222fa97-serviceca\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:05:28.995509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995457    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-bin\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.995509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995494    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-certificates\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.995734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995523    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6zhx\" (UniqueName: \"kubernetes.io/projected/34ca0f75-5a97-4496-a7ae-060723b2d625-kube-api-access-s6zhx\") pod \"kas-connection-checker-685895c7c9-xmv5g\" (UID: \"34ca0f75-5a97-4496-a7ae-060723b2d625\") " pod="kube-system/kas-connection-checker-685895c7c9-xmv5g"
Nov 02 22:05:28.995734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995551    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovn-node-metrics-cert\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.995734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995525    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-stats-auth\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:28.995734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995585    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:28.995734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995614    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-netd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.995734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995642    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:28.995734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995667    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/daa20bd0-d6e1-4044-812a-37ef99a19dac-config\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:28.995734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995698    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-var-lib-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.995734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995704    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-certificates\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.996071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995763    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-script-lib\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.996071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995804    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:28.996071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995857    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2t4v\" (UniqueName: \"kubernetes.io/projected/2dc1992e-41ec-4bde-9211-47162c8fb3c2-kube-api-access-t2t4v\") pod \"volume-data-source-validator-58c9f5f46d-xptqw\" (UID: \"2dc1992e-41ec-4bde-9211-47162c8fb3c2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw"
Nov 02 22:05:28.996071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995887    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-snapshots\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:28.996071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995896    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/daa20bd0-d6e1-4044-812a-37ef99a19dac-config\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:28.996071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.995932    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-service-ca-bundle\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:28.996071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996005    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b66dw\" (UniqueName: \"kubernetes.io/projected/3465159d-8024-48a2-8408-eb1d1db96a8a-kube-api-access-b66dw\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:28.996071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996028    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovnkube-script-lib\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.996071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996052    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21f54242-5bab-4162-a435-85a080f8685c-tmp-dir\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:05:28.996071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996059    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-service-ca-bundle\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996079    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996113    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3465159d-8024-48a2-8408-eb1d1db96a8a-snapshots\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996120    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8759d3be-8ddd-4719-808f-3611fcd7f887-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996128    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21f54242-5bab-4162-a435-85a080f8685c-tmp-dir\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996154    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996189    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7wvw\" (UniqueName: \"kubernetes.io/projected/daa20bd0-d6e1-4044-812a-37ef99a19dac-kube-api-access-q7wvw\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996208    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-ovn\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996233    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f32e923-7f5f-4ba7-b44d-6441fc46f876-serving-cert\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996262    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996287    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdx65\" (UniqueName: \"kubernetes.io/projected/21f54242-5bab-4162-a435-85a080f8685c-kube-api-access-jdx65\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996324    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8759d3be-8ddd-4719-808f-3611fcd7f887-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996325    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqt4z\" (UniqueName: \"kubernetes.io/projected/2c712bb5-ab02-4ae7-8d21-b2274448ccf3-kube-api-access-cqt4z\") pod \"network-check-source-6f78d59f99-4t246\" (UID: \"2c712bb5-ab02-4ae7-8d21-b2274448ccf3\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4t246"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996360    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-netns\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996407    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-trusted-ca\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996481    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-image-registry-private-configuration\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996551    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:28.996605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996573    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:28.997226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996607    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/624e4d57-0db4-4be3-b512-e5065d800d01-nginx-conf\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:28.997226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996643    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21f54242-5bab-4162-a435-85a080f8685c-hosts-file\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:05:28.997226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996675    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-default-certificate\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:28.997226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996692    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25h79\" (UniqueName: \"kubernetes.io/projected/8f32e923-7f5f-4ba7-b44d-6441fc46f876-kube-api-access-25h79\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:28.997226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996719    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f32e923-7f5f-4ba7-b44d-6441fc46f876-trusted-ca\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:28.997226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996737    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:28.997226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996891    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f32e923-7f5f-4ba7-b44d-6441fc46f876-serving-cert\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:28.997226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996935    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3465159d-8024-48a2-8408-eb1d1db96a8a-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:28.997226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996958    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/624e4d57-0db4-4be3-b512-e5065d800d01-nginx-conf\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:28.997226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:28.996945    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-default-certificate\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:29.000557 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.000541    5067 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 22:05:29.048523 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048456    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 22:05:29.048523 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048512    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 22:05:29.048668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048540    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 22:05:29.048668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048608    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-4gtmj\""
Nov 02 22:05:29.048781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048700    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 02 22:05:29.048781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048714    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 02 22:05:29.048781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048747    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 02 22:05:29.048781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048779    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 22:05:29.048952 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048780    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 02 22:05:29.048952 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048820    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-l6kn9\""
Nov 02 22:05:29.048952 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048829    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 02 22:05:29.048952 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048900    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 02 22:05:29.049094 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.048953    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-656w2\""
Nov 02 22:05:29.049094 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049076    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-98gpk\""
Nov 02 22:05:29.049179 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049151    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v45jr\""
Nov 02 22:05:29.049227 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049201    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4ts5p\""
Nov 02 22:05:29.049275 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049237    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 22:05:29.049275 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049266    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 22:05:29.049359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049279    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rsfpb\""
Nov 02 22:05:29.049359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049284    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4rxp8\""
Nov 02 22:05:29.049359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049285    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rksbh\""
Nov 02 22:05:29.049359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049285    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zcntw\""
Nov 02 22:05:29.049359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049311    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fcg7g\""
Nov 02 22:05:29.049359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049308    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-6jxx8\""
Nov 02 22:05:29.049933 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049908    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 22:05:29.049933 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049927    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bsff7\""
Nov 02 22:05:29.049933 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049932    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zz2sg\""
Nov 02 22:05:29.050107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049950    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-xh86t\""
Nov 02 22:05:29.050107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049958    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-rrhfb\""
Nov 02 22:05:29.050107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049973    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fkc8k\""
Nov 02 22:05:29.050107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.049991    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-g764w\""
Nov 02 22:05:29.050107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.050008    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gwrjd\""
Nov 02 22:05:29.050107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.050020    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t4sw7\""
Nov 02 22:05:29.050107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.050027    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-klqgw\""
Nov 02 22:05:29.050348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.050132    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 02 22:05:29.050348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.050153    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l4knj\""
Nov 02 22:05:29.050348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.050184    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 22:05:29.050348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.050229    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q5sg6\""
Nov 02 22:05:29.054540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.054526    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3465159d-8024-48a2-8408-eb1d1db96a8a-serving-cert\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:29.055707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.055691    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-installation-pull-secrets\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:29.055806 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.055793    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cfdf849-3285-4c0c-bcc8-2401667a460b-ovn-node-metrics-cert\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.056633 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.056619    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9fkg\" (UniqueName: \"kubernetes.io/projected/dde46c9c-2501-4545-918c-9a3af3a6b381-kube-api-access-w9fkg\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:29.056720 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.056704    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:29.056766 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.056711    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-image-registry-private-configuration\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:29.057232 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.057216    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsh4k\" (UniqueName: \"kubernetes.io/projected/9cfdf849-3285-4c0c-bcc8-2401667a460b-kube-api-access-tsh4k\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.057789 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.057774    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-bound-sa-token\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:29.060101 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.060069    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c94vr\" (UniqueName: \"kubernetes.io/projected/1e801619-9294-420b-8ad5-cd1005832f16-kube-api-access-c94vr\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:05:29.060187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.060104    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6zhx\" (UniqueName: \"kubernetes.io/projected/34ca0f75-5a97-4496-a7ae-060723b2d625-kube-api-access-s6zhx\") pod \"kas-connection-checker-685895c7c9-xmv5g\" (UID: \"34ca0f75-5a97-4496-a7ae-060723b2d625\") " pod="kube-system/kas-connection-checker-685895c7c9-xmv5g"
Nov 02 22:05:29.061003 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.060502    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfbdb\" (UniqueName: \"kubernetes.io/projected/012f51ba-d2ba-4d6b-b90d-67fe89601119-kube-api-access-sfbdb\") pod \"kas-connection-checker-685895c7c9-7vddh\" (UID: \"012f51ba-d2ba-4d6b-b90d-67fe89601119\") " pod="kube-system/kas-connection-checker-685895c7c9-7vddh"
Nov 02 22:05:29.061003 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.060632    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j2wx\" (UniqueName: \"kubernetes.io/projected/101f50b9-bfd1-405a-ade4-6e1cc683dfda-kube-api-access-8j2wx\") pod \"kas-connection-checker-685895c7c9-tjhsr\" (UID: \"101f50b9-bfd1-405a-ade4-6e1cc683dfda\") " pod="kube-system/kas-connection-checker-685895c7c9-tjhsr"
Nov 02 22:05:29.061003 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.060851    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2t4v\" (UniqueName: \"kubernetes.io/projected/2dc1992e-41ec-4bde-9211-47162c8fb3c2-kube-api-access-t2t4v\") pod \"volume-data-source-validator-58c9f5f46d-xptqw\" (UID: \"2dc1992e-41ec-4bde-9211-47162c8fb3c2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xptqw"
Nov 02 22:05:29.061003 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.060905    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25h79\" (UniqueName: \"kubernetes.io/projected/8f32e923-7f5f-4ba7-b44d-6441fc46f876-kube-api-access-25h79\") pod \"console-operator-7cd56b8dbc-b5mdw\" (UID: \"8f32e923-7f5f-4ba7-b44d-6441fc46f876\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:29.061003 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.060954    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b66dw\" (UniqueName: \"kubernetes.io/projected/3465159d-8024-48a2-8408-eb1d1db96a8a-kube-api-access-b66dw\") pod \"insights-operator-7c4986d449-9tsq4\" (UID: \"3465159d-8024-48a2-8408-eb1d1db96a8a\") " pod="openshift-insights/insights-operator-7c4986d449-9tsq4"
Nov 02 22:05:29.061003 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.060965    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdx65\" (UniqueName: \"kubernetes.io/projected/21f54242-5bab-4162-a435-85a080f8685c-kube-api-access-jdx65\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:05:29.061245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.061024    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x47x\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-kube-api-access-7x47x\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:29.061245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.061051    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfdlq\" (UniqueName: \"kubernetes.io/projected/35ec85a5-34bf-4cd2-b209-7744c6f51a20-kube-api-access-lfdlq\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:29.061979 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.061958    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmjvr\" (UniqueName: \"kubernetes.io/projected/3505867a-1caf-4cef-b570-841afd5f7d85-kube-api-access-dmjvr\") pod \"network-check-target-7j964\" (UID: \"3505867a-1caf-4cef-b570-841afd5f7d85\") " pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:29.062065 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.062017    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqjht\" (UniqueName: \"kubernetes.io/projected/8759d3be-8ddd-4719-808f-3611fcd7f887-kube-api-access-zqjht\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:29.063751 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.063727    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7wvw\" (UniqueName: \"kubernetes.io/projected/daa20bd0-d6e1-4044-812a-37ef99a19dac-kube-api-access-q7wvw\") pod \"service-ca-operator-544d749c8c-r6sk8\" (UID: \"daa20bd0-d6e1-4044-812a-37ef99a19dac\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8"
Nov 02 22:05:29.064435 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.064420    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqt4z\" (UniqueName: \"kubernetes.io/projected/2c712bb5-ab02-4ae7-8d21-b2274448ccf3-kube-api-access-cqt4z\") pod \"network-check-source-6f78d59f99-4t246\" (UID: \"2c712bb5-ab02-4ae7-8d21-b2274448ccf3\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4t246"
Nov 02 22:05:29.065237 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.065221    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stbfv\" (UniqueName: \"kubernetes.io/projected/a2c3d84e-0830-4800-bb72-28bd2b6ca4c5-kube-api-access-stbfv\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7\" (UID: \"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7"
Nov 02 22:05:29.065300 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.065284    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dqqc\" (UniqueName: \"kubernetes.io/projected/713aba87-e440-4d2f-a339-52b3f222fa97-kube-api-access-8dqqc\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:05:29.097359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097337    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-cabundle\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:29.097457 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097382    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-hostroot\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.097457 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097400    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxf2x\" (UniqueName: \"kubernetes.io/projected/d987c59d-73b7-4883-b686-49809eeb776b-kube-api-access-lxf2x\") pod \"migrator-66df5d496b-m78f7\" (UID: \"d987c59d-73b7-4883-b686-49809eeb776b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7"
Nov 02 22:05:29.097457 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097423    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dx64\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-kube-api-access-2dx64\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.097553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097466    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:29.097553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097485    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.097553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097500    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfaa3942-3844-475e-9d7a-95bf64101115-tmp-dir\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:29.097553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097517    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-kubernetes\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.097553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097534    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-sys\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.097747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097560    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-device-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.097747 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.097594    5067 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:29.097747 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.097615    5067 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b78b7d88-kdfn8: secret "image-registry-tls" not found
Nov 02 22:05:29.097747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097629    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4843105e-92f4-4b83-83fd-06e7a217e009-agent-certs\") pod \"konnectivity-agent-ww4zl\" (UID: \"4843105e-92f4-4b83-83fd-06e7a217e009\") " pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:05:29.097747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097600    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfaa3942-3844-475e-9d7a-95bf64101115-tmp-dir\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:29.097747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097662    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.097747 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.097685    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls podName:a8b9eeac-3e3f-4612-a696-2ede201d83ac nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.597668067 +0000 UTC m=+1.881477747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls") pod "image-registry-66b78b7d88-kdfn8" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac") : secret "image-registry-tls" not found
Nov 02 22:05:29.098066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097752    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.098066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097808    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.098066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097819    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-cabundle\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:29.098066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097882    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-cni-binary-copy\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.098066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097924    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gls6g\" (UniqueName: \"kubernetes.io/projected/f020dd86-76b6-4433-a902-632d15269b91-kube-api-access-gls6g\") pod \"ingress-canary-9pqwd\" (UID: \"f020dd86-76b6-4433-a902-632d15269b91\") " pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:29.098066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097943    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-modprobe-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.098066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097954    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4843105e-92f4-4b83-83fd-06e7a217e009-agent-certs\") pod \"konnectivity-agent-ww4zl\" (UID: \"4843105e-92f4-4b83-83fd-06e7a217e009\") " pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:05:29.098066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097967    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.098066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.097998    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-netns\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.098066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098060    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.098460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098094    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-cni-binary-copy\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.098460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098104    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-run-netns\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.098460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098105    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ltdh\" (UniqueName: \"kubernetes.io/projected/bfaa3942-3844-475e-9d7a-95bf64101115-kube-api-access-7ltdh\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:29.098460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098141    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-registration-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.098460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098160    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-dbus\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:29.098460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098199    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.098629 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098619    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-system-cni-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.098662 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098645    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-socket-dir-parent\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.098691 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098667    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.098691 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098683    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dq8x\" (UniqueName: \"kubernetes.io/projected/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-kube-api-access-5dq8x\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.098765 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098699    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.098765 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098717    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:29.098765 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098733    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-multus-daemon-config\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.098872 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098782    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-slash\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.098872 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098833    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.098940 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.098929    5067 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 22:05:29.098988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098964    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-bin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.098988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098978    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0caf91c1-266c-405f-92b4-070981db24c3-multus-daemon-config\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.099083 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.098984    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-slash\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099083 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.098991    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert podName:624e4d57-0db4-4be3-b512-e5065d800d01 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.598975367 +0000 UTC m=+1.882785044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-x67tb" (UID: "624e4d57-0db4-4be3-b512-e5065d800d01") : secret "networking-console-plugin-cert" not found
Nov 02 22:05:29.099083 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099042    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nff6m\" (UniqueName: \"kubernetes.io/projected/fc25374f-cb3c-45ce-a523-af6fa11b570e-kube-api-access-nff6m\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.099083 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099068    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-etc-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099096    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-kubelet-config\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:29.099229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099121    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-cnibin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.099229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099145    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-etc-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099156    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4843105e-92f4-4b83-83fd-06e7a217e009-konnectivity-ca\") pod \"konnectivity-agent-ww4zl\" (UID: \"4843105e-92f4-4b83-83fd-06e7a217e009\") " pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:05:29.099229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099219    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099313    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099340    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-sys-fs\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.099386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099380    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r82z\" (UniqueName: \"kubernetes.io/projected/0caf91c1-266c-405f-92b4-070981db24c3-kube-api-access-4r82z\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.099490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099385    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099386    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099406    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4843105e-92f4-4b83-83fd-06e7a217e009-konnectivity-ca\") pod \"konnectivity-agent-ww4zl\" (UID: \"4843105e-92f4-4b83-83fd-06e7a217e009\") " pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:05:29.099490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099410    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-node-log\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099445    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-node-log\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099636 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099545    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-bin\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099636 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099574    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-k8s-cni-cncf-io\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.099636 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099604    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-bin\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.099636 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099625    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-systemd\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.099752 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099664    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-tmp\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.099752 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099698    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:29.099752 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099715    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:29.099843 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099756    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-tmp\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.099843 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.099763    5067 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 22:05:29.099843 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099783    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.099843 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.099817    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls podName:35ec85a5-34bf-4cd2-b209-7744c6f51a20 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.599798161 +0000 UTC m=+1.883607841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-xf2zt" (UID: "35ec85a5-34bf-4cd2-b209-7744c6f51a20") : secret "samples-operator-tls" not found
Nov 02 22:05:29.099843 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099830    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-netd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099849    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-run\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099867    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-tuned\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099887    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-cni-netd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099917    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-var-lib-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099945    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-tuned\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099951    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-service-ca-bundle\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099971    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-var-lib-openvswitch\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.099980    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6f7s\" (UniqueName: \"kubernetes.io/projected/6b703071-5807-41b5-afe0-a2c85e23af47-kube-api-access-d6f7s\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100015    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-image-registry-private-configuration\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100042    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-installation-pull-secrets\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100064    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-service-ca-bundle\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:29.100077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100076    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100127    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cnibin\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100163    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.100168    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.600154859 +0000 UTC m=+1.883964541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : configmap references non-existent config key: service-ca.crt
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100204    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100236    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-installation-pull-secrets\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100252    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-image-registry-private-configuration\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100259    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-archives-path\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100277    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-ovn\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100300    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-ovn\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100306    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100344    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-archives-path\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100344    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfaa3942-3844-475e-9d7a-95bf64101115-config-volume\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100398    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100414    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-var-lib-kubelet\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100433    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtlnb\" (UniqueName: \"kubernetes.io/projected/56b1aa47-8106-416e-869d-bfc5c10ef795-kube-api-access-jtlnb\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100450    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e47f56ea-3a73-4ff0-965d-3a21319aef2e-original-pull-secret\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:29.100477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100472    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100499    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-etc-selinux\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100527    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21f54242-5bab-4162-a435-85a080f8685c-hosts-file\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100530    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfaa3942-3844-475e-9d7a-95bf64101115-config-volume\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100555    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-system-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100592    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/21f54242-5bab-4162-a435-85a080f8685c-hosts-file\") pod \"node-resolver-t5mqk\" (UID: \"21f54242-5bab-4162-a435-85a080f8685c\") " pod="openshift-dns/node-resolver-t5mqk"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100620    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-multus-certs\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100659    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-multus\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100684    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/985b7a7f-8711-4eec-8732-0fedb158f7a4-data-volume\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100705    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8b6z\" (UniqueName: \"kubernetes.io/projected/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-api-access-v8b6z\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100720    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f020dd86-76b6-4433-a902-632d15269b91-cert\") pod \"ingress-canary-9pqwd\" (UID: \"f020dd86-76b6-4433-a902-632d15269b91\") " pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100733    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-etc-kubernetes\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100761    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e801619-9294-420b-8ad5-cd1005832f16-host-slash\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100785    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100801    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/985b7a7f-8711-4eec-8732-0fedb158f7a4-data-volume\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100822    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/713aba87-e440-4d2f-a339-52b3f222fa97-host\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100841    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e801619-9294-420b-8ad5-cd1005832f16-host-slash\") pod \"iptables-alerter-bpxkm\" (UID: \"1e801619-9294-420b-8ad5-cd1005832f16\") " pod="openshift-network-operator/iptables-alerter-bpxkm"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100860    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-trusted-ca\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100877    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/713aba87-e440-4d2f-a339-52b3f222fa97-host\") pod \"node-ca-v76fm\" (UID: \"713aba87-e440-4d2f-a339-52b3f222fa97\") " pod="openshift-image-registry/node-ca-v76fm"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100890    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b45cw\" (UniqueName: \"kubernetes.io/projected/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-kube-api-access-b45cw\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100914    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f020dd86-76b6-4433-a902-632d15269b91-cert\") pod \"ingress-canary-9pqwd\" (UID: \"f020dd86-76b6-4433-a902-632d15269b91\") " pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100920    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x97rj\" (UniqueName: \"kubernetes.io/projected/1d714b69-b66b-46b4-8520-3537a5138ce3-kube-api-access-x97rj\") pod \"migrator-66df5d496b-ljl56\" (UID: \"1d714b69-b66b-46b4-8520-3537a5138ce3\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.100950    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-kubelet\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.100988 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.100951    5067 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101010    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-conf-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101105    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/985b7a7f-8711-4eec-8732-0fedb158f7a4-crio-socket\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101149    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-bound-sa-token\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.101179    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.601162141 +0000 UTC m=+1.884971834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : secret "router-metrics-certs-default" not found
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101215    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-systemd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101250    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-key\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101253    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-run-systemd\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101324    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-kubelet\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101350    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-log-socket\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101397    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-host-kubelet\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101398    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101426    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2fe1c794-551a-40bf-b71a-4dc069aa0875-signing-key\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101438    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-systemd-units\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.101462    5067 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101472    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-netns\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101482    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-systemd-units\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101497    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-conf\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.101510    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls podName:8759d3be-8ddd-4719-808f-3611fcd7f887 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.601494704 +0000 UTC m=+1.885304385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-rjgqf" (UID: "8759d3be-8ddd-4719-808f-3611fcd7f887") : secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101475    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cfdf849-3285-4c0c-bcc8-2401667a460b-log-socket\") pod \"ovnkube-node-5jljx\" (UID: \"9cfdf849-3285-4c0c-bcc8-2401667a460b\") " pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101567    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e6866c0-b229-4cd8-b977-32d366cced05-ca-trust-extracted\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101599    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-trusted-ca\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101630    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-registry-certificates\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101653    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e6866c0-b229-4cd8-b977-32d366cced05-ca-trust-extracted\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101663    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101691    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwsml\" (UniqueName: \"kubernetes.io/projected/2fe1c794-551a-40bf-b71a-4dc069aa0875-kube-api-access-qwsml\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101717    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysconfig\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101724    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-registry-certificates\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101763    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-lib-modules\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.101777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101790    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-host\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.102925 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101819    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-trusted-ca-bundle\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:29.102925 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.101863    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-socket-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.102925 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.102416    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-trusted-ca-bundle\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:29.154927 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.154892    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ltdh\" (UniqueName: \"kubernetes.io/projected/bfaa3942-3844-475e-9d7a-95bf64101115-kube-api-access-7ltdh\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:29.155074 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.155023    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dx64\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-kube-api-access-2dx64\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.155074 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.155051    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gls6g\" (UniqueName: \"kubernetes.io/projected/f020dd86-76b6-4433-a902-632d15269b91-kube-api-access-gls6g\") pod \"ingress-canary-9pqwd\" (UID: \"f020dd86-76b6-4433-a902-632d15269b91\") " pod="openshift-ingress-canary/ingress-canary-9pqwd"
Nov 02 22:05:29.155145 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.155113    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxf2x\" (UniqueName: \"kubernetes.io/projected/d987c59d-73b7-4883-b686-49809eeb776b-kube-api-access-lxf2x\") pod \"migrator-66df5d496b-m78f7\" (UID: \"d987c59d-73b7-4883-b686-49809eeb776b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7"
Nov 02 22:05:29.156022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.156004    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtlnb\" (UniqueName: \"kubernetes.io/projected/56b1aa47-8106-416e-869d-bfc5c10ef795-kube-api-access-jtlnb\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:29.156646 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.156630    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r82z\" (UniqueName: \"kubernetes.io/projected/0caf91c1-266c-405f-92b4-070981db24c3-kube-api-access-4r82z\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.156973 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.156956    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dq8x\" (UniqueName: \"kubernetes.io/projected/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-kube-api-access-5dq8x\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.162279 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.162259    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nff6m\" (UniqueName: \"kubernetes.io/projected/fc25374f-cb3c-45ce-a523-af6fa11b570e-kube-api-access-nff6m\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.162482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.162466    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6f7s\" (UniqueName: \"kubernetes.io/projected/6b703071-5807-41b5-afe0-a2c85e23af47-kube-api-access-d6f7s\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.164287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.164268    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-bound-sa-token\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.164506 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.164487    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x97rj\" (UniqueName: \"kubernetes.io/projected/1d714b69-b66b-46b4-8520-3537a5138ce3-kube-api-access-x97rj\") pod \"migrator-66df5d496b-ljl56\" (UID: \"1d714b69-b66b-46b4-8520-3537a5138ce3\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ljl56"
Nov 02 22:05:29.164557 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.164539    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b45cw\" (UniqueName: \"kubernetes.io/projected/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-kube-api-access-b45cw\") pod \"periodic-gathering-qzwrq-r69mk\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") " pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:05:29.165248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.165232    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwsml\" (UniqueName: \"kubernetes.io/projected/2fe1c794-551a-40bf-b71a-4dc069aa0875-kube-api-access-qwsml\") pod \"service-ca-5fffc97bdf-2fhhl\" (UID: \"2fe1c794-551a-40bf-b71a-4dc069aa0875\") " pod="openshift-service-ca/service-ca-5fffc97bdf-2fhhl"
Nov 02 22:05:29.166709 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.166693    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8b6z\" (UniqueName: \"kubernetes.io/projected/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-api-access-v8b6z\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.191790 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.191765    5067 scope.go:122] "RemoveContainer" containerID="8b54386b91d0c86d7932e0617cc2334f012249e97c574493e3d810f242944686"
Nov 02 22:05:29.203134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203106    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-modprobe-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.203235 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203149    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.203298 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.203246    5067 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:29.203298 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.203260    5067 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56dc494c89-qz46b: secret "image-registry-tls" not found
Nov 02 22:05:29.203298 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203265    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-modprobe-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.203462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203303    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-registration-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.203462 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.203320    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls podName:1e6866c0-b229-4cd8-b977-32d366cced05 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.70330145 +0000 UTC m=+1.987111117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls") pod "image-registry-56dc494c89-qz46b" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05") : secret "image-registry-tls" not found
Nov 02 22:05:29.203462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203346    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-dbus\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:29.203462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203388    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-system-cni-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.203462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203411    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-socket-dir-parent\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.203462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203425    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-system-cni-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.203462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203388    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-registration-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.203462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203442    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-dbus\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:29.203462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203437    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.203484    5067 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203503    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-socket-dir-parent\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203507    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-bin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.203524    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.703510888 +0000 UTC m=+1.987320553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : configmap "kube-rbac-proxy" not found
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203542    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-bin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203562    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-kubelet-config\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203588    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-cnibin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203615    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-sys-fs\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203642    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-k8s-cni-cncf-io\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203648    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-cnibin\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203666    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-systemd\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203684    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-k8s-cni-cncf-io\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203686    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e47f56ea-3a73-4ff0-965d-3a21319aef2e-kubelet-config\") pod \"global-pull-secret-syncer-78snq\" (UID: \"e47f56ea-3a73-4ff0-965d-3a21319aef2e\") " pod="kube-system/global-pull-secret-syncer-78snq"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203689    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203729    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-sys-fs\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203736    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-systemd\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203743    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.203814    5067 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203852    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-run\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.203877 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.203873    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.703856824 +0000 UTC m=+1.987666506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203919    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cnibin\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203947    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203968    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-run\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204011    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-cnibin\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.203974    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204102    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204128    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-var-lib-kubelet\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204136    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-d\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204169    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204160    5067 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204195    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-etc-selinux\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204199    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204235    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-system-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204245    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-var-lib-kubelet\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204259    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-multus-certs\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204297    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-etc-selinux\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204285    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-multus\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204342    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-etc-kubernetes\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204394    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2bc1cd9-8631-4e6e-82be-053f3d8df4d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xbz67\" (UID: \"e2bc1cd9-8631-4e6e-82be-053f3d8df4d4\") " pod="openshift-multus/multus-additional-cni-plugins-xbz67"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204400    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-multus-certs\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204342    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204456    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-etc-kubernetes\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204441    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-system-cni-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204473    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-cni-multus\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204484    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-kubelet\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204511    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-conf-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204543    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/985b7a7f-8711-4eec-8732-0fedb158f7a4-crio-socket\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204579    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-multus-conf-dir\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204591    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-netns\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204625    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/985b7a7f-8711-4eec-8732-0fedb158f7a4-crio-socket\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204641    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-run-netns\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204627    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-conf\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204577    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-host-var-lib-kubelet\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204684    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204710    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysconfig\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204733    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-lib-modules\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204755    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-host\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204780    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-socket-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.204791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204803    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-hostroot\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204823    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysconfig\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204839    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-lib-modules\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204756    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-sysctl-conf\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204859    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0caf91c1-266c-405f-92b4-070981db24c3-hostroot\") pod \"multus-dkbhl\" (UID: \"0caf91c1-266c-405f-92b4-070981db24c3\") " pod="openshift-multus/multus-dkbhl"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.204875    5067 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204882    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-host\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204912    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-socket-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.204966    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-kubernetes\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.205000    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-sys\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.205035    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-device-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.205079    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-etc-kubernetes\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.205112    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls podName:bfaa3942-3844-475e-9d7a-95bf64101115 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:29.705094026 +0000 UTC m=+1.988903710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls") pod "dns-default-gd729" (UID: "bfaa3942-3844-475e-9d7a-95bf64101115") : secret "dns-default-metrics-tls" not found
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.205134    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc25374f-cb3c-45ce-a523-af6fa11b570e-sys\") pod \"tuned-f4n2z\" (UID: \"fc25374f-cb3c-45ce-a523-af6fa11b570e\") " pod="openshift-cluster-node-tuning-operator/tuned-f4n2z"
Nov 02 22:05:29.206154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.205116    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b703071-5807-41b5-afe0-a2c85e23af47-device-dir\") pod \"aws-ebs-csi-driver-node-4sdmr\" (UID: \"6b703071-5807-41b5-afe0-a2c85e23af47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4sdmr"
Nov 02 22:05:29.206597 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.206568    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56b1aa47-8106-416e-869d-bfc5c10ef795-metrics-certs\") pod \"network-metrics-daemon-69fj9\" (UID: \"56b1aa47-8106-416e-869d-bfc5c10ef795\") " pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:29.504240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.504208    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-69fj9"
Nov 02 22:05:29.610709 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.610672    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:29.610838 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.610739    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:29.610838 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.610809    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:29.610838 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.610815    5067 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:29.611047 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.610855    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:29.611047 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.610876    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls podName:8759d3be-8ddd-4719-808f-3611fcd7f887 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:30.610860572 +0000 UTC m=+2.894670237 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-rjgqf" (UID: "8759d3be-8ddd-4719-808f-3611fcd7f887") : secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:29.611047 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.610882    5067 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:29.611047 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.610897    5067 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b78b7d88-kdfn8: secret "image-registry-tls" not found
Nov 02 22:05:29.611047 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.610923    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:29.611047 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.610932    5067 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 22:05:29.611047 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.610955    5067 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 22:05:29.611047 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.610959    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls podName:a8b9eeac-3e3f-4612-a696-2ede201d83ac nodeName:}" failed. No retries permitted until 2026-11-02 22:05:30.610944581 +0000 UTC m=+2.894754267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls") pod "image-registry-66b78b7d88-kdfn8" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac") : secret "image-registry-tls" not found
Nov 02 22:05:29.611047 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.611029    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls podName:35ec85a5-34bf-4cd2-b209-7744c6f51a20 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:30.611013783 +0000 UTC m=+2.894823448 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-xf2zt" (UID: "35ec85a5-34bf-4cd2-b209-7744c6f51a20") : secret "samples-operator-tls" not found
Nov 02 22:05:29.611047 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.611049    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:30.611040514 +0000 UTC m=+2.894850183 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : configmap references non-existent config key: service-ca.crt
Nov 02 22:05:29.611331 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.611084    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert podName:624e4d57-0db4-4be3-b512-e5065d800d01 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:30.611057988 +0000 UTC m=+2.894867656 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-x67tb" (UID: "624e4d57-0db4-4be3-b512-e5065d800d01") : secret "networking-console-plugin-cert" not found
Nov 02 22:05:29.611331 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.611184    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:29.611443 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.611354    5067 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 22:05:29.611443 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.611425    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:30.61141238 +0000 UTC m=+2.895222060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : secret "router-metrics-certs-default" not found
Nov 02 22:05:29.712325 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.712285    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:29.712522 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.712387    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:29.712522 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.712426    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.712522 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.712450    5067 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 22:05:29.712522 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.712494    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:29.712522 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.712517    5067 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:29.712780 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.712535    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls podName:bfaa3942-3844-475e-9d7a-95bf64101115 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:30.71251991 +0000 UTC m=+2.996329593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls") pod "dns-default-gd729" (UID: "bfaa3942-3844-475e-9d7a-95bf64101115") : secret "dns-default-metrics-tls" not found
Nov 02 22:05:29.712780 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.712535    5067 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56dc494c89-qz46b: secret "image-registry-tls" not found
Nov 02 22:05:29.712780 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.712583    5067 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 22:05:29.712780 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.712592    5067 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:29.712780 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.712606    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls podName:1e6866c0-b229-4cd8-b977-32d366cced05 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:30.712588899 +0000 UTC m=+2.996398568 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls") pod "image-registry-56dc494c89-qz46b" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05") : secret "image-registry-tls" not found
Nov 02 22:05:29.712780 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.712634    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:30.7126177 +0000 UTC m=+2.996427369 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : configmap "kube-rbac-proxy" not found
Nov 02 22:05:29.712780 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:29.712653    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:30.712643943 +0000 UTC m=+2.996453614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:29.901975 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.901885    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-69fj9"]
Nov 02 22:05:29.908428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.908407    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:05:29.908549 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.908515    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw" event={"ID":"8f32e923-7f5f-4ba7-b44d-6441fc46f876","Type":"ContainerStarted","Data":"edfea9f1c5d9fc2fe117f2407665065a76cd9f436716eb3954cbac72c27ee11f"}
Nov 02 22:05:29.909623 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:29.909594    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b1aa47_8106_416e_869d_bfc5c10ef795.slice/crio-08c9ab8beba30206540efb826a5fc167c74894aa28acb9b356d0c2e5e3986779 WatchSource:0}: Error finding container 08c9ab8beba30206540efb826a5fc167c74894aa28acb9b356d0c2e5e3986779: Status 404 returned error can't find the container with id 08c9ab8beba30206540efb826a5fc167c74894aa28acb9b356d0c2e5e3986779
Nov 02 22:05:29.910127 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.910104    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-m78f7" event={"ID":"d987c59d-73b7-4883-b686-49809eeb776b","Type":"ContainerStarted","Data":"5dea19504be16471c9e6a63cc2a6b03b2d0e1ae1fd6ec2420e0c19bb38cec537"}
Nov 02 22:05:29.911556 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:29.911540    5067 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:05:30.468591 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.468554    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:30.473436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.473414    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-b5mdw"
Nov 02 22:05:30.621795 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.621754    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:30.621990 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.621819    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:30.621990 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.621932    5067 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 22:05:30.621990 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.621967    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:30.622138 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.622008    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert podName:624e4d57-0db4-4be3-b512-e5065d800d01 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.621986614 +0000 UTC m=+4.905796293 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-x67tb" (UID: "624e4d57-0db4-4be3-b512-e5065d800d01") : secret "networking-console-plugin-cert" not found
Nov 02 22:05:30.622138 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.622070    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:30.622138 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.622119    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:30.622282 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.622178    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:30.622282 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.622194    5067 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 22:05:30.622282 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.622216    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.62220022 +0000 UTC m=+4.906009900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : configmap references non-existent config key: service-ca.crt
Nov 02 22:05:30.622282 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.622250    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.622235557 +0000 UTC m=+4.906045240 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : secret "router-metrics-certs-default" not found
Nov 02 22:05:30.622282 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.622261    5067 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:30.622282 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.622272    5067 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66b78b7d88-kdfn8: secret "image-registry-tls" not found
Nov 02 22:05:30.622282 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.622269    5067 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:30.622578 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.622307    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls podName:a8b9eeac-3e3f-4612-a696-2ede201d83ac nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.62229487 +0000 UTC m=+4.906104553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls") pod "image-registry-66b78b7d88-kdfn8" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac") : secret "image-registry-tls" not found
Nov 02 22:05:30.622578 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.622338    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls podName:8759d3be-8ddd-4719-808f-3611fcd7f887 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.622314794 +0000 UTC m=+4.906124464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-rjgqf" (UID: "8759d3be-8ddd-4719-808f-3611fcd7f887") : secret "cluster-monitoring-operator-tls" not found
Nov 02 22:05:30.624897 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.624873    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ec85a5-34bf-4cd2-b209-7744c6f51a20-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-xf2zt\" (UID: \"35ec85a5-34bf-4cd2-b209-7744c6f51a20\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:30.692475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.692439    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"
Nov 02 22:05:30.722768 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.722736    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:30.722898 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.722811    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:30.722898 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.722859    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:30.723006 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.722926    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:30.723006 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.722955    5067 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 22:05:30.723006 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.722977    5067 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 22:05:30.723155 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.722996    5067 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56dc494c89-qz46b: secret "image-registry-tls" not found
Nov 02 22:05:30.723155 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.723027    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls podName:bfaa3942-3844-475e-9d7a-95bf64101115 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.723007515 +0000 UTC m=+5.006817183 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls") pod "dns-default-gd729" (UID: "bfaa3942-3844-475e-9d7a-95bf64101115") : secret "dns-default-metrics-tls" not found
Nov 02 22:05:30.723155 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.723073    5067 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:30.723155 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.723140    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls podName:1e6866c0-b229-4cd8-b977-32d366cced05 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.72306765 +0000 UTC m=+5.006877333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls") pod "image-registry-56dc494c89-qz46b" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05") : secret "image-registry-tls" not found
Nov 02 22:05:30.723413 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:30.723172    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:32.723150814 +0000 UTC m=+5.006960495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:30.723578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.723408    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/985b7a7f-8711-4eec-8732-0fedb158f7a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:30.915470 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:30.915436    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-69fj9" event={"ID":"56b1aa47-8106-416e-869d-bfc5c10ef795","Type":"ContainerStarted","Data":"08c9ab8beba30206540efb826a5fc167c74894aa28acb9b356d0c2e5e3986779"}
Nov 02 22:05:31.086208 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:31.086139    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt"]
Nov 02 22:05:31.902900 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:31.902871    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t5mqk_21f54242-5bab-4162-a435-85a080f8685c/dns-node-resolver/0.log"
Nov 02 22:05:31.920764 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:31.920732    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt" event={"ID":"35ec85a5-34bf-4cd2-b209-7744c6f51a20","Type":"ContainerStarted","Data":"948d3d8c1d271eca3ebe1e0141a4337e5ca9e87869a1640e22f0a5f40512f980"}
Nov 02 22:05:32.359572 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.359544    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:32.641814 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.641776    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:32.642008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.641842    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:32.642008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.641894    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:32.642008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.641935    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:32.642008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.641974    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:32.642197 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:32.642053    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:36.642030564 +0000 UTC m=+8.925840246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : configmap references non-existent config key: service-ca.crt
Nov 02 22:05:32.642408 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:32.642347    5067 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 22:05:32.642488 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:32.642427    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:36.642408606 +0000 UTC m=+8.926218294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : secret "router-metrics-certs-default" not found
Nov 02 22:05:32.642488 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:32.642456    5067 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 22:05:32.642614 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:32.642522    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert podName:624e4d57-0db4-4be3-b512-e5065d800d01 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:36.642506965 +0000 UTC m=+8.926316635 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-x67tb" (UID: "624e4d57-0db4-4be3-b512-e5065d800d01") : secret "networking-console-plugin-cert" not found
Nov 02 22:05:32.644554 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.644519    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8759d3be-8ddd-4719-808f-3611fcd7f887-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-rjgqf\" (UID: \"8759d3be-8ddd-4719-808f-3611fcd7f887\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:32.644695 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.644673    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"image-registry-66b78b7d88-kdfn8\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") " pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:32.743230 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.743189    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:32.743475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.743261    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:32.743475 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:32.743329    5067 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 22:05:32.743475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.743326    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:32.743475 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:32.743433    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls podName:bfaa3942-3844-475e-9d7a-95bf64101115 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:36.743412268 +0000 UTC m=+9.027221950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls") pod "dns-default-gd729" (UID: "bfaa3942-3844-475e-9d7a-95bf64101115") : secret "dns-default-metrics-tls" not found
Nov 02 22:05:32.743727 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:32.743475    5067 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:32.743727 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:32.743537    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:36.743517666 +0000 UTC m=+9.027327336 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:32.745822 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.745801    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"image-registry-56dc494c89-qz46b\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") " pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:32.791458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.791412    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"
Nov 02 22:05:32.793582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.793557    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:32.802476 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.802448    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:32.927741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.927706    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-69fj9" event={"ID":"56b1aa47-8106-416e-869d-bfc5c10ef795","Type":"ContainerStarted","Data":"96fed4ef7af29fb50e8b3f7c57c2442479897b1d8ede4cc53692a4277be172c4"}
Nov 02 22:05:32.928044 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:32.927774    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-69fj9" event={"ID":"56b1aa47-8106-416e-869d-bfc5c10ef795","Type":"ContainerStarted","Data":"96cef06b58ea949b0ad95ccf1e3fb494229a57b7a1ac5002a1dc24a6ece85243"}
Nov 02 22:05:33.033428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.033400    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v76fm_713aba87-e440-4d2f-a339-52b3f222fa97/node-ca/0.log"
Nov 02 22:05:33.231267 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.231230    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf"]
Nov 02 22:05:33.235637 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:33.235599    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8759d3be_8ddd_4719_808f_3611fcd7f887.slice/crio-e983ec5b30644e67b888b9460f6802a9cbd5592b382655859751ef9697930b84 WatchSource:0}: Error finding container e983ec5b30644e67b888b9460f6802a9cbd5592b382655859751ef9697930b84: Status 404 returned error can't find the container with id e983ec5b30644e67b888b9460f6802a9cbd5592b382655859751ef9697930b84
Nov 02 22:05:33.246464 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.246396    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66b78b7d88-kdfn8"]
Nov 02 22:05:33.250470 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:33.250443    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b9eeac_3e3f_4612_a696_2ede201d83ac.slice/crio-cce090deeb333ef2879fec7212e9a5ebe9cc41505942793391a703208d5c99ed WatchSource:0}: Error finding container cce090deeb333ef2879fec7212e9a5ebe9cc41505942793391a703208d5c99ed: Status 404 returned error can't find the container with id cce090deeb333ef2879fec7212e9a5ebe9cc41505942793391a703208d5c99ed
Nov 02 22:05:33.284727 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.284698    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56dc494c89-qz46b"]
Nov 02 22:05:33.289690 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:33.289472    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e6866c0_b229_4cd8_b977_32d366cced05.slice/crio-8f1fc6ad6e7eff6077f474717820d6a8dadc7a4e93fc852bac325f63d1f288c3 WatchSource:0}: Error finding container 8f1fc6ad6e7eff6077f474717820d6a8dadc7a4e93fc852bac325f63d1f288c3: Status 404 returned error can't find the container with id 8f1fc6ad6e7eff6077f474717820d6a8dadc7a4e93fc852bac325f63d1f288c3
Nov 02 22:05:33.433868 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.433838    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9pqwd_f020dd86-76b6-4433-a902-632d15269b91/serve-healthcheck-canary/0.log"
Nov 02 22:05:33.633629 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.633555    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-ljl56_1d714b69-b66b-46b4-8520-3537a5138ce3/migrator/0.log"
Nov 02 22:05:33.833307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.833271    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-m78f7_d987c59d-73b7-4883-b686-49809eeb776b/migrator/1.log"
Nov 02 22:05:33.933059 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.932971    5067 generic.go:356] "Generic (PLEG): container finished" podID="daa20bd0-d6e1-4044-812a-37ef99a19dac" containerID="80e82ea7e935c62c94c0e03149cf851afc3da705eb840dd2b9fbe2abe5a5d233" exitCode=0
Nov 02 22:05:33.933059 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.933034    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8" event={"ID":"daa20bd0-d6e1-4044-812a-37ef99a19dac","Type":"ContainerDied","Data":"80e82ea7e935c62c94c0e03149cf851afc3da705eb840dd2b9fbe2abe5a5d233"}
Nov 02 22:05:33.933555 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.933451    5067 scope.go:122] "RemoveContainer" containerID="80e82ea7e935c62c94c0e03149cf851afc3da705eb840dd2b9fbe2abe5a5d233"
Nov 02 22:05:33.935159 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.935134    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt" event={"ID":"35ec85a5-34bf-4cd2-b209-7744c6f51a20","Type":"ContainerStarted","Data":"2d4da62752a3161901370f766fb801f3bc33ca8aa17a42d4df8ed10e8dc99bce"}
Nov 02 22:05:33.935273 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.935188    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-xf2zt" event={"ID":"35ec85a5-34bf-4cd2-b209-7744c6f51a20","Type":"ContainerStarted","Data":"2e0e3f6f6679856d42e41150b7af05f70f0a69aad6306446ccf94e40f33726b0"}
Nov 02 22:05:33.936412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.936388    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf" event={"ID":"8759d3be-8ddd-4719-808f-3611fcd7f887","Type":"ContainerStarted","Data":"e983ec5b30644e67b888b9460f6802a9cbd5592b382655859751ef9697930b84"}
Nov 02 22:05:33.937916 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.937896    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56dc494c89-qz46b" event={"ID":"1e6866c0-b229-4cd8-b977-32d366cced05","Type":"ContainerStarted","Data":"11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31"}
Nov 02 22:05:33.938166 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.937934    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56dc494c89-qz46b" event={"ID":"1e6866c0-b229-4cd8-b977-32d366cced05","Type":"ContainerStarted","Data":"8f1fc6ad6e7eff6077f474717820d6a8dadc7a4e93fc852bac325f63d1f288c3"}
Nov 02 22:05:33.939617 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.939597    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8" event={"ID":"a8b9eeac-3e3f-4612-a696-2ede201d83ac","Type":"ContainerStarted","Data":"a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d"}
Nov 02 22:05:33.939707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:33.939640    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8" event={"ID":"a8b9eeac-3e3f-4612-a696-2ede201d83ac","Type":"ContainerStarted","Data":"cce090deeb333ef2879fec7212e9a5ebe9cc41505942793391a703208d5c99ed"}
Nov 02 22:05:34.034275 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:34.034250    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7_a2c3d84e-0830-4800-bb72-28bd2b6ca4c5/kube-storage-version-migrator-operator/0.log"
Nov 02 22:05:34.946442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:34.946409    5067 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 22:05:34.946993 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:34.946472    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-r6sk8" event={"ID":"daa20bd0-d6e1-4044-812a-37ef99a19dac","Type":"ContainerStarted","Data":"4b8461aa88b4623c832991c7bf44a91eea4ecf6054301e298ff5c62ae1dc69c7"}
Nov 02 22:05:34.946993 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:34.946414    5067 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 22:05:35.394344 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:35.394309    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:35.394567 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:35.394545    5067 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 22:05:35.638015 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:35.637983    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:36.118921 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.118883    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:36.685377 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.685324    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:36.685549 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.685449    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:36.685549 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.685496    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:36.685549 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:36.685498    5067 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 22:05:36.685655 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:36.685576    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs podName:dde46c9c-2501-4545-918c-9a3af3a6b381 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:44.685556559 +0000 UTC m=+16.969366230 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs") pod "router-default-b448bfb57-lttxh" (UID: "dde46c9c-2501-4545-918c-9a3af3a6b381") : secret "router-metrics-certs-default" not found
Nov 02 22:05:36.686314 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.686283    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde46c9c-2501-4545-918c-9a3af3a6b381-service-ca-bundle\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:36.688168 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.688145    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/624e4d57-0db4-4be3-b512-e5065d800d01-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-x67tb\" (UID: \"624e4d57-0db4-4be3-b512-e5065d800d01\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:36.697007 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.696986    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb"
Nov 02 22:05:36.786692 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.786660    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:36.786808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.786726    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:36.786864 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:36.786851    5067 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:36.786923 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:05:36.786900    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls podName:985b7a7f-8711-4eec-8732-0fedb158f7a4 nodeName:}" failed. No retries permitted until 2026-11-02 22:05:44.786888607 +0000 UTC m=+17.070698284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fdcvz" (UID: "985b7a7f-8711-4eec-8732-0fedb158f7a4") : secret "insights-runtime-extractor-tls" not found
Nov 02 22:05:36.789705 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.789650    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfaa3942-3844-475e-9d7a-95bf64101115-metrics-tls\") pod \"dns-default-gd729\" (UID: \"bfaa3942-3844-475e-9d7a-95bf64101115\") " pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:36.957667 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:36.957587    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-rjgqf" event={"ID":"8759d3be-8ddd-4719-808f-3611fcd7f887","Type":"ContainerStarted","Data":"dd15f702df288fd9e37d1bf413ac26a5de7b4e31fac9cc151c58364bad847435"}
Nov 02 22:05:37.005249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:37.005213    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:37.029687 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:37.029653    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5jljx"
Nov 02 22:05:37.082020 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:37.081730    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-x67tb"]
Nov 02 22:05:37.123467 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:37.123438    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624e4d57_0db4_4be3_b512_e5065d800d01.slice/crio-48f72d74a71461ff6e4b1a3e0d4ae49d19d6bfe9065c4b82750e08a90adfb90b WatchSource:0}: Error finding container 48f72d74a71461ff6e4b1a3e0d4ae49d19d6bfe9065c4b82750e08a90adfb90b: Status 404 returned error can't find the container with id 48f72d74a71461ff6e4b1a3e0d4ae49d19d6bfe9065c4b82750e08a90adfb90b
Nov 02 22:05:37.400187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:37.400154    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gd729"]
Nov 02 22:05:37.404801 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:37.404775    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfaa3942_3844_475e_9d7a_95bf64101115.slice/crio-837133f26327c78943e0afe08209677d6e300ce3f0b99d3b2837741ddb6db2b2 WatchSource:0}: Error finding container 837133f26327c78943e0afe08209677d6e300ce3f0b99d3b2837741ddb6db2b2: Status 404 returned error can't find the container with id 837133f26327c78943e0afe08209677d6e300ce3f0b99d3b2837741ddb6db2b2
Nov 02 22:05:37.962637 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:37.962590    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb" event={"ID":"624e4d57-0db4-4be3-b512-e5065d800d01","Type":"ContainerStarted","Data":"48f72d74a71461ff6e4b1a3e0d4ae49d19d6bfe9065c4b82750e08a90adfb90b"}
Nov 02 22:05:37.963979 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:37.963913    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gd729" event={"ID":"bfaa3942-3844-475e-9d7a-95bf64101115","Type":"ContainerStarted","Data":"837133f26327c78943e0afe08209677d6e300ce3f0b99d3b2837741ddb6db2b2"}
Nov 02 22:05:39.973505 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:39.973465    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-x67tb" event={"ID":"624e4d57-0db4-4be3-b512-e5065d800d01","Type":"ContainerStarted","Data":"de509166bc29606cda6f0828f6ec09ee7d95331000c4259527fcbfec15b65c9b"}
Nov 02 22:05:40.903095 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:40.903063    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:41.135761 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:41.135728    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ww4zl"
Nov 02 22:05:42.985951 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:42.985913    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gd729" event={"ID":"bfaa3942-3844-475e-9d7a-95bf64101115","Type":"ContainerStarted","Data":"24a8899bcc7c99ff30bfce44ccb7415625524539577d92a1455745e25d1108cc"}
Nov 02 22:05:42.986391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:42.985993    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gd729" event={"ID":"bfaa3942-3844-475e-9d7a-95bf64101115","Type":"ContainerStarted","Data":"24bacd6cf11ec9eefe430d9bb010e3fdc8e0934fd60c488f0d2fc9410e6f681a"}
Nov 02 22:05:42.986391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:42.986048    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:44.760097 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:44.760062    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:44.762460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:44.762438    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde46c9c-2501-4545-918c-9a3af3a6b381-metrics-certs\") pod \"router-default-b448bfb57-lttxh\" (UID: \"dde46c9c-2501-4545-918c-9a3af3a6b381\") " pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:44.791169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:44.791132    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:44.861537 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:44.861503    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:44.864744 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:44.864717    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/985b7a7f-8711-4eec-8732-0fedb158f7a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fdcvz\" (UID: \"985b7a7f-8711-4eec-8732-0fedb158f7a4\") " pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:45.107258 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.107167    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fdcvz"
Nov 02 22:05:45.206687 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.206657    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b448bfb57-lttxh"]
Nov 02 22:05:45.209897 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:45.209861    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde46c9c_2501_4545_918c_9a3af3a6b381.slice/crio-58c8031a1c078773dfac0c9171256b2e413638391395b36076b598859c64362b WatchSource:0}: Error finding container 58c8031a1c078773dfac0c9171256b2e413638391395b36076b598859c64362b: Status 404 returned error can't find the container with id 58c8031a1c078773dfac0c9171256b2e413638391395b36076b598859c64362b
Nov 02 22:05:45.471028 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.470990    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fdcvz"]
Nov 02 22:05:45.474727 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:45.474699    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985b7a7f_8711_4eec_8732_0fedb158f7a4.slice/crio-7c2662abe7171077489db4f08928877aff079383d005727f31edfa0d4b852355 WatchSource:0}: Error finding container 7c2662abe7171077489db4f08928877aff079383d005727f31edfa0d4b852355: Status 404 returned error can't find the container with id 7c2662abe7171077489db4f08928877aff079383d005727f31edfa0d4b852355
Nov 02 22:05:45.662444 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.662412    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-wzpg7"]
Nov 02 22:05:45.665479 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.665452    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg"]
Nov 02 22:05:45.665633 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.665614    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-wzpg7"
Nov 02 22:05:45.670288 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.669043    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg"
Nov 02 22:05:45.683500 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.683475    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66b78b7d88-kdfn8"]
Nov 02 22:05:45.714380 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.714345    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 02 22:05:45.714679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.714659    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-flvrj\""
Nov 02 22:05:45.739792 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.739720    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 02 22:05:45.739911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.739822    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-djw8z\""
Nov 02 22:05:45.739956 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.739935    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 02 22:05:45.754554 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.754526    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg"]
Nov 02 22:05:45.756198 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.756174    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-wzpg7"]
Nov 02 22:05:45.762303 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.762267    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-854b6cc9c8-b7q6r"]
Nov 02 22:05:45.764883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.764862    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.768616 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.768594    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0cbcf6f0-083c-43f2-bb04-bab241e8f748-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-2rgrg\" (UID: \"0cbcf6f0-083c-43f2-bb04-bab241e8f748\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg"
Nov 02 22:05:45.768720 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.768638    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1-tmp\") pod \"downloads-6dd789c8b7-wzpg7\" (UID: \"f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1\") " pod="openshift-console/downloads-6dd789c8b7-wzpg7"
Nov 02 22:05:45.768789 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.768745    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7plr\" (UniqueName: \"kubernetes.io/projected/f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1-kube-api-access-x7plr\") pod \"downloads-6dd789c8b7-wzpg7\" (UID: \"f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1\") " pod="openshift-console/downloads-6dd789c8b7-wzpg7"
Nov 02 22:05:45.824267 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.824238    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-854b6cc9c8-b7q6r"]
Nov 02 22:05:45.869943 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.869907    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/208186a2-d662-42b3-98d5-67f1f7db6338-registry-certificates\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.869943 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.869940    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/208186a2-d662-42b3-98d5-67f1f7db6338-trusted-ca\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.870136 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.869969    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drhvl\" (UniqueName: \"kubernetes.io/projected/208186a2-d662-42b3-98d5-67f1f7db6338-kube-api-access-drhvl\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.870136 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.870086    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7plr\" (UniqueName: \"kubernetes.io/projected/f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1-kube-api-access-x7plr\") pod \"downloads-6dd789c8b7-wzpg7\" (UID: \"f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1\") " pod="openshift-console/downloads-6dd789c8b7-wzpg7"
Nov 02 22:05:45.870136 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.870119    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/208186a2-d662-42b3-98d5-67f1f7db6338-ca-trust-extracted\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.870260 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.870137    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/208186a2-d662-42b3-98d5-67f1f7db6338-bound-sa-token\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.870260 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.870164    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0cbcf6f0-083c-43f2-bb04-bab241e8f748-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-2rgrg\" (UID: \"0cbcf6f0-083c-43f2-bb04-bab241e8f748\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg"
Nov 02 22:05:45.870260 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.870187    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/208186a2-d662-42b3-98d5-67f1f7db6338-installation-pull-secrets\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.870509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.870286    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/208186a2-d662-42b3-98d5-67f1f7db6338-image-registry-private-configuration\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.870509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.870319    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1-tmp\") pod \"downloads-6dd789c8b7-wzpg7\" (UID: \"f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1\") " pod="openshift-console/downloads-6dd789c8b7-wzpg7"
Nov 02 22:05:45.870509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.870337    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/208186a2-d662-42b3-98d5-67f1f7db6338-registry-tls\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.870705 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.870686    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1-tmp\") pod \"downloads-6dd789c8b7-wzpg7\" (UID: \"f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1\") " pod="openshift-console/downloads-6dd789c8b7-wzpg7"
Nov 02 22:05:45.872655 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.872631    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0cbcf6f0-083c-43f2-bb04-bab241e8f748-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-2rgrg\" (UID: \"0cbcf6f0-083c-43f2-bb04-bab241e8f748\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg"
Nov 02 22:05:45.928285 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.928253    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7plr\" (UniqueName: \"kubernetes.io/projected/f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1-kube-api-access-x7plr\") pod \"downloads-6dd789c8b7-wzpg7\" (UID: \"f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1\") " pod="openshift-console/downloads-6dd789c8b7-wzpg7"
Nov 02 22:05:45.970896 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.970865    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/208186a2-d662-42b3-98d5-67f1f7db6338-registry-certificates\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.970896 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.970900    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/208186a2-d662-42b3-98d5-67f1f7db6338-trusted-ca\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.971197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.970928    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drhvl\" (UniqueName: \"kubernetes.io/projected/208186a2-d662-42b3-98d5-67f1f7db6338-kube-api-access-drhvl\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.971197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.970974    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/208186a2-d662-42b3-98d5-67f1f7db6338-ca-trust-extracted\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.971197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.971134    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/208186a2-d662-42b3-98d5-67f1f7db6338-bound-sa-token\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.971197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.971178    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/208186a2-d662-42b3-98d5-67f1f7db6338-installation-pull-secrets\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.971461 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.971215    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/208186a2-d662-42b3-98d5-67f1f7db6338-image-registry-private-configuration\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.971461 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.971283    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/208186a2-d662-42b3-98d5-67f1f7db6338-registry-tls\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.971461 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.971359    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/208186a2-d662-42b3-98d5-67f1f7db6338-ca-trust-extracted\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.971808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.971784    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/208186a2-d662-42b3-98d5-67f1f7db6338-registry-certificates\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.972241 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.972220    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/208186a2-d662-42b3-98d5-67f1f7db6338-trusted-ca\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.973732 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.973711    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/208186a2-d662-42b3-98d5-67f1f7db6338-image-registry-private-configuration\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.973888 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.973867    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/208186a2-d662-42b3-98d5-67f1f7db6338-installation-pull-secrets\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.973935 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.973879    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/208186a2-d662-42b3-98d5-67f1f7db6338-registry-tls\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:45.983764 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.983741    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-wzpg7"
Nov 02 22:05:45.989191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.989169    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg"
Nov 02 22:05:45.999123 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.999088    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fdcvz" event={"ID":"985b7a7f-8711-4eec-8732-0fedb158f7a4","Type":"ContainerStarted","Data":"cb9a660468e9839e137b939d6f8fa129f28b4f3c5cbab067b0307e97e0ebc476"}
Nov 02 22:05:45.999229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:45.999164    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fdcvz" event={"ID":"985b7a7f-8711-4eec-8732-0fedb158f7a4","Type":"ContainerStarted","Data":"7c2662abe7171077489db4f08928877aff079383d005727f31edfa0d4b852355"}
Nov 02 22:05:46.000861 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:46.000833    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b448bfb57-lttxh" event={"ID":"dde46c9c-2501-4545-918c-9a3af3a6b381","Type":"ContainerStarted","Data":"827d81212e8ec6de4851aca57626607e7493e7bbef251b01a2fbc78f6792512c"}
Nov 02 22:05:46.000948 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:46.000900    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b448bfb57-lttxh" event={"ID":"dde46c9c-2501-4545-918c-9a3af3a6b381","Type":"ContainerStarted","Data":"58c8031a1c078773dfac0c9171256b2e413638391395b36076b598859c64362b"}
Nov 02 22:05:46.030546 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:46.030511    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drhvl\" (UniqueName: \"kubernetes.io/projected/208186a2-d662-42b3-98d5-67f1f7db6338-kube-api-access-drhvl\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:46.031019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:46.030995    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/208186a2-d662-42b3-98d5-67f1f7db6338-bound-sa-token\") pod \"image-registry-854b6cc9c8-b7q6r\" (UID: \"208186a2-d662-42b3-98d5-67f1f7db6338\") " pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:46.086596 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:46.086564    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:46.355862 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:46.355779    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-wzpg7"]
Nov 02 22:05:46.359414 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:46.359385    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5dfa2df_0acd_4e6e_8eb2_03aa9044cfd1.slice/crio-f2527d317216137982da93659eb5b2b26b4aa02ac800c13dc217679dbc7fb875 WatchSource:0}: Error finding container f2527d317216137982da93659eb5b2b26b4aa02ac800c13dc217679dbc7fb875: Status 404 returned error can't find the container with id f2527d317216137982da93659eb5b2b26b4aa02ac800c13dc217679dbc7fb875
Nov 02 22:05:46.404656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:46.404623    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg"]
Nov 02 22:05:46.408844 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:46.408809    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cbcf6f0_083c_43f2_bb04_bab241e8f748.slice/crio-48a3bdb6a29de180602eb8490a1837677076484de14a59110a908a0f7098b721 WatchSource:0}: Error finding container 48a3bdb6a29de180602eb8490a1837677076484de14a59110a908a0f7098b721: Status 404 returned error can't find the container with id 48a3bdb6a29de180602eb8490a1837677076484de14a59110a908a0f7098b721
Nov 02 22:05:46.469595 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:46.469564    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-854b6cc9c8-b7q6r"]
Nov 02 22:05:46.472940 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:46.472909    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod208186a2_d662_42b3_98d5_67f1f7db6338.slice/crio-5c9c8b26264c1931138730449d768fad22f2fc5a9fdbd1e38d3cc49720c0bd7d WatchSource:0}: Error finding container 5c9c8b26264c1931138730449d768fad22f2fc5a9fdbd1e38d3cc49720c0bd7d: Status 404 returned error can't find the container with id 5c9c8b26264c1931138730449d768fad22f2fc5a9fdbd1e38d3cc49720c0bd7d
Nov 02 22:05:46.791677 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:46.791590    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:46.797040 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:46.797014    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:47.006212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:47.006176    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r" event={"ID":"208186a2-d662-42b3-98d5-67f1f7db6338","Type":"ContainerStarted","Data":"446add8a2e1a85ae8ef8d584db82a2f3332e8f0ec701191d217f0c716dcc5f89"}
Nov 02 22:05:47.006403 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:47.006265    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r" event={"ID":"208186a2-d662-42b3-98d5-67f1f7db6338","Type":"ContainerStarted","Data":"5c9c8b26264c1931138730449d768fad22f2fc5a9fdbd1e38d3cc49720c0bd7d"}
Nov 02 22:05:47.006403 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:47.006292    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:05:47.007484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:47.007460    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg" event={"ID":"0cbcf6f0-083c-43f2-bb04-bab241e8f748","Type":"ContainerStarted","Data":"48a3bdb6a29de180602eb8490a1837677076484de14a59110a908a0f7098b721"}
Nov 02 22:05:47.008638 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:47.008613    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-wzpg7" event={"ID":"f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1","Type":"ContainerStarted","Data":"f2527d317216137982da93659eb5b2b26b4aa02ac800c13dc217679dbc7fb875"}
Nov 02 22:05:47.010252 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:47.010227    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fdcvz" event={"ID":"985b7a7f-8711-4eec-8732-0fedb158f7a4","Type":"ContainerStarted","Data":"0c9fbff8b8da1f266ce707887e7d45708c13c3ac5aec8a8163c597bcc0a5c52c"}
Nov 02 22:05:47.010455 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:47.010432    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:47.011744 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:47.011726    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-b448bfb57-lttxh"
Nov 02 22:05:47.123718 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:47.123612    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r" podStartSLOduration=2.123599614 podStartE2EDuration="2.123599614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:05:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:47.122865189 +0000 UTC m=+19.406674901" watchObservedRunningTime="2026-11-02 22:05:47.123599614 +0000 UTC m=+19.407409298"
Nov 02 22:05:48.035627 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:48.035582    5067 patch_prober.go:29] interesting pod/image-registry-56dc494c89-qz46b container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 02 22:05:48.036066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:48.035668    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56dc494c89-qz46b" podUID="1e6866c0-b229-4cd8-b977-32d366cced05" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:05:49.024471 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:49.024441    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg" event={"ID":"0cbcf6f0-083c-43f2-bb04-bab241e8f748","Type":"ContainerStarted","Data":"c69e088621676e638fa199ebe02c559535c48c3d1336632d43bcb77791fc250f"}
Nov 02 22:05:49.025015 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:49.024982    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg"
Nov 02 22:05:49.030351 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:49.030328    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg"
Nov 02 22:05:49.133098 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:49.133045    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2rgrg" podStartSLOduration=2.409229565 podStartE2EDuration="4.133028615s" totalImagesPullingTime="1.72379905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:05:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:05:49.132276132 +0000 UTC m=+21.416085819" watchObservedRunningTime="2026-11-02 22:05:49.133028615 +0000 UTC m=+21.416838298"
Nov 02 22:05:50.032724 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:50.032676    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fdcvz" event={"ID":"985b7a7f-8711-4eec-8732-0fedb158f7a4","Type":"ContainerStarted","Data":"f865e9e2d73d2c6cc6c525869ad27768d1ec39f3f70507bc0a54ac0ab17e36e7"}
Nov 02 22:05:52.991908 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:52.991876    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gd729"
Nov 02 22:05:55.641525 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:55.641489    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7j964"
Nov 02 22:05:55.689750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:55.689720    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:05:56.123870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:56.123841    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:05:57.626355 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.624608    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6ljgp"]
Nov 02 22:05:57.653360 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.653327    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.702429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.702386    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 02 22:05:57.748877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.748827    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 02 22:05:57.748877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.748869    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 02 22:05:57.749169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.748965    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 02 22:05:57.749169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.749090    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v5dnx\""
Nov 02 22:05:57.788719 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.788681    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-wtmp\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.788906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.788733    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-metrics-client-ca\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.788906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.788831    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdcmg\" (UniqueName: \"kubernetes.io/projected/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-kube-api-access-jdcmg\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.788906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.788883    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.789057 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.788952    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.789057 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.788982    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-root\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.789121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.789098    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-sys\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.789175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.789136    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-textfile\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.789224 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.789176    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-tls\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.889956    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-sys\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890011    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-textfile\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890050    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-tls\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890081    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-sys\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890085    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-wtmp\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890155    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-metrics-client-ca\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890194    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdcmg\" (UniqueName: \"kubernetes.io/projected/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-kube-api-access-jdcmg\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890228    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-wtmp\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890230    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890275    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890305    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-root\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890435    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-root\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.891613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.890600    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-textfile\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.892880 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.892850    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.892880 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.892875    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-metrics-client-ca\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.902747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.902721    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.902862 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.902771    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-node-exporter-tls\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:57.949598 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:57.949569    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdcmg\" (UniqueName: \"kubernetes.io/projected/4c9ece87-d259-49f9-bad1-f9573fe2ebf1-kube-api-access-jdcmg\") pod \"node-exporter-6ljgp\" (UID: \"4c9ece87-d259-49f9-bad1-f9573fe2ebf1\") " pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:58.026179 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:58.026133    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6ljgp"
Nov 02 22:05:58.068563 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:05:58.068530    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c9ece87_d259_49f9_bad1_f9573fe2ebf1.slice/crio-cebfc5a0430bc1d0fa0b32135bc2b948a6a7f5f3160d51d6593fa674ca2da120 WatchSource:0}: Error finding container cebfc5a0430bc1d0fa0b32135bc2b948a6a7f5f3160d51d6593fa674ca2da120: Status 404 returned error can't find the container with id cebfc5a0430bc1d0fa0b32135bc2b948a6a7f5f3160d51d6593fa674ca2da120
Nov 02 22:05:59.078010 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:05:59.077962    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ljgp" event={"ID":"4c9ece87-d259-49f9-bad1-f9573fe2ebf1","Type":"ContainerStarted","Data":"cebfc5a0430bc1d0fa0b32135bc2b948a6a7f5f3160d51d6593fa674ca2da120"}
Nov 02 22:06:01.091207 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:01.091171    5067 generic.go:356] "Generic (PLEG): container finished" podID="4c9ece87-d259-49f9-bad1-f9573fe2ebf1" containerID="f22b2a29ecb27ea6a98a23248213a2a8c26781010585bd3469a2f32f0acde996" exitCode=0
Nov 02 22:06:01.091744 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:01.091286    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ljgp" event={"ID":"4c9ece87-d259-49f9-bad1-f9573fe2ebf1","Type":"ContainerDied","Data":"f22b2a29ecb27ea6a98a23248213a2a8c26781010585bd3469a2f32f0acde996"}
Nov 02 22:06:02.098566 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:02.098518    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ljgp" event={"ID":"4c9ece87-d259-49f9-bad1-f9573fe2ebf1","Type":"ContainerStarted","Data":"d12e1dcb1e74161202d1cbc1f46f4690f161c418e00dbb981a913d1e09c593ed"}
Nov 02 22:06:02.099010 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:02.098604    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6ljgp" event={"ID":"4c9ece87-d259-49f9-bad1-f9573fe2ebf1","Type":"ContainerStarted","Data":"0e6291efa91caef8a1cc1c082bb8ee6b9de5b735ef809bca261b4ac5903b7fb5"}
Nov 02 22:06:02.250443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:02.250378    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6ljgp" podStartSLOduration=2.90429045 podStartE2EDuration="5.250343324s" totalImagesPullingTime="2.256641871s" totalInitContainerRuntime="89.411003ms" isStatefulPod=true podCreationTimestamp="2026-11-02 22:05:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:06:02.214200863 +0000 UTC m=+34.498010568" watchObservedRunningTime="2026-11-02 22:06:02.250343324 +0000 UTC m=+34.534153010"
Nov 02 22:06:04.110206 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:04.110163    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-wzpg7" event={"ID":"f5dfa2df-0acd-4e6e-8eb2-03aa9044cfd1","Type":"ContainerStarted","Data":"c63839000f94a6697a6f4afef409543efe277ad159043c8363e2fe7ee5cf48ce"}
Nov 02 22:06:04.110750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:04.110417    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-wzpg7"
Nov 02 22:06:04.127191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:04.127160    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-wzpg7"
Nov 02 22:06:04.314547 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:04.314487    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-wzpg7" podStartSLOduration=2.562639185 podStartE2EDuration="19.314469109s" totalImagesPullingTime="16.751829924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:05:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:06:04.266764177 +0000 UTC m=+36.550573895" watchObservedRunningTime="2026-11-02 22:06:04.314469109 +0000 UTC m=+36.598278796"
Nov 02 22:06:07.579827 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:06:07.579784    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:06:08.021843 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:08.021815    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-854b6cc9c8-b7q6r"
Nov 02 22:06:08.232479 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:08.232442    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56dc494c89-qz46b"]
Nov 02 22:06:10.704858 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:10.704771    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8" podUID="a8b9eeac-3e3f-4612-a696-2ede201d83ac" containerName="registry" containerID="cri-o://a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d" gracePeriod=30
Nov 02 22:06:11.013416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.013357    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:06:11.093241 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.093202    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8b9eeac-3e3f-4612-a696-2ede201d83ac-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8b9eeac-3e3f-4612-a696-2ede201d83ac-ca-trust-extracted\") pod \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") "
Nov 02 22:06:11.093439 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.093261    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-bound-sa-token\") pod \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") "
Nov 02 22:06:11.093439 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.093302    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-kube-api-access-7x47x\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-kube-api-access-7x47x\") pod \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") "
Nov 02 22:06:11.093439 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.093336    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") pod \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") "
Nov 02 22:06:11.093439 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.093374    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-certificates\") pod \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") "
Nov 02 22:06:11.093439 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.093432    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-installation-pull-secrets\") pod \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") "
Nov 02 22:06:11.093703 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.093472    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-image-registry-private-configuration\") pod \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") "
Nov 02 22:06:11.093703 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.093536    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-trusted-ca\") pod \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\" (UID: \"a8b9eeac-3e3f-4612-a696-2ede201d83ac\") "
Nov 02 22:06:11.094202 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.093841    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-certificates" pod "a8b9eeac-3e3f-4612-a696-2ede201d83ac" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:06:11.094321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.094251    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-trusted-ca" pod "a8b9eeac-3e3f-4612-a696-2ede201d83ac" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:06:11.098458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.098429    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-bound-sa-token" pod "a8b9eeac-3e3f-4612-a696-2ede201d83ac" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:06:11.098678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.098644    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-installation-pull-secrets" pod "a8b9eeac-3e3f-4612-a696-2ede201d83ac" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:06:11.098788 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.098744    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-kube-api-access-7x47x" pod "a8b9eeac-3e3f-4612-a696-2ede201d83ac" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac"). InnerVolumeSpecName "kube-api-access-7x47x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:06:11.098852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.098803    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-image-registry-private-configuration" pod "a8b9eeac-3e3f-4612-a696-2ede201d83ac" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:06:11.098852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.098815    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls" pod "a8b9eeac-3e3f-4612-a696-2ede201d83ac" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:06:11.103210 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.103183    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b9eeac-3e3f-4612-a696-2ede201d83ac-ca-trust-extracted" pod "a8b9eeac-3e3f-4612-a696-2ede201d83ac" (UID: "a8b9eeac-3e3f-4612-a696-2ede201d83ac"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:06:11.154274 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.154237    5067 generic.go:356] "Generic (PLEG): container finished" podID="a8b9eeac-3e3f-4612-a696-2ede201d83ac" containerID="a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d" exitCode=0
Nov 02 22:06:11.154475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.154300    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8"
Nov 02 22:06:11.154475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.154312    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8" event={"ID":"a8b9eeac-3e3f-4612-a696-2ede201d83ac","Type":"ContainerDied","Data":"a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d"}
Nov 02 22:06:11.154475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.154389    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66b78b7d88-kdfn8" event={"ID":"a8b9eeac-3e3f-4612-a696-2ede201d83ac","Type":"ContainerDied","Data":"cce090deeb333ef2879fec7212e9a5ebe9cc41505942793391a703208d5c99ed"}
Nov 02 22:06:11.154475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.154420    5067 scope.go:122] "RemoveContainer" containerID="a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d"
Nov 02 22:06:11.165156 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.165136    5067 scope.go:122] "RemoveContainer" containerID="a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d"
Nov 02 22:06:11.165470 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:06:11.165438    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d\": container with ID starting with a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d not found: ID does not exist" containerID="a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d"
Nov 02 22:06:11.165585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.165481    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d"} err="failed to get container status \"a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d\": rpc error: code = NotFound desc = could not find container \"a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d\": container with ID starting with a96a2d6517e4271121a2481c26dbb181aae105d795a1321e83c1180a01afb59d not found: ID does not exist"
Nov 02 22:06:11.195158 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.195123    5067 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-trusted-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:11.195158 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.195156    5067 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8b9eeac-3e3f-4612-a696-2ede201d83ac-ca-trust-extracted\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:11.195398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.195171    5067 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-bound-sa-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:11.195398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.195183    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x47x\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-kube-api-access-7x47x\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:11.195398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.195194    5067 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-tls\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:11.195398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.195206    5067 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8b9eeac-3e3f-4612-a696-2ede201d83ac-registry-certificates\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:11.195398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.195219    5067 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-installation-pull-secrets\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:11.195398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.195232    5067 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8b9eeac-3e3f-4612-a696-2ede201d83ac-image-registry-private-configuration\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:11.293303 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.293231    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66b78b7d88-kdfn8"]
Nov 02 22:06:11.296984 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.296952    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66b78b7d88-kdfn8"]
Nov 02 22:06:11.844806 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:11.844758    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8b9eeac-3e3f-4612-a696-2ede201d83ac" path="/var/lib/kubelet/pods/a8b9eeac-3e3f-4612-a696-2ede201d83ac/volumes"
Nov 02 22:06:17.310424 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:17.310382    5067 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ww4zl_4843105e-92f4-4b83-83fd-06e7a217e009/konnectivity-agent/0.log" limit=1024
Nov 02 22:06:17.718302 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:17.718271    5067 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ww4zl_4843105e-92f4-4b83-83fd-06e7a217e009/konnectivity-agent/0.log" limit=1024
Nov 02 22:06:17.973058 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:17.972975    5067 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ww4zl_4843105e-92f4-4b83-83fd-06e7a217e009/konnectivity-agent/0.log" limit=1024
Nov 02 22:06:33.256012 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.255949    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56dc494c89-qz46b" podUID="1e6866c0-b229-4cd8-b977-32d366cced05" containerName="registry" containerID="cri-o://11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31" gracePeriod=30
Nov 02 22:06:33.541602 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.541536    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:06:33.690430 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.690395    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-image-registry-private-configuration\") pod \"1e6866c0-b229-4cd8-b977-32d366cced05\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") "
Nov 02 22:06:33.690623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.690448    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-kube-api-access-2dx64\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-kube-api-access-2dx64\") pod \"1e6866c0-b229-4cd8-b977-32d366cced05\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") "
Nov 02 22:06:33.690623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.690486    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-registry-certificates\") pod \"1e6866c0-b229-4cd8-b977-32d366cced05\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") "
Nov 02 22:06:33.690623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.690525    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-installation-pull-secrets\") pod \"1e6866c0-b229-4cd8-b977-32d366cced05\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") "
Nov 02 22:06:33.690623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.690583    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e6866c0-b229-4cd8-b977-32d366cced05-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e6866c0-b229-4cd8-b977-32d366cced05-ca-trust-extracted\") pod \"1e6866c0-b229-4cd8-b977-32d366cced05\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") "
Nov 02 22:06:33.690623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.690611    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-bound-sa-token\") pod \"1e6866c0-b229-4cd8-b977-32d366cced05\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") "
Nov 02 22:06:33.691064 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.690672    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") pod \"1e6866c0-b229-4cd8-b977-32d366cced05\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") "
Nov 02 22:06:33.691064 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.690731    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-trusted-ca\") pod \"1e6866c0-b229-4cd8-b977-32d366cced05\" (UID: \"1e6866c0-b229-4cd8-b977-32d366cced05\") "
Nov 02 22:06:33.691064 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.690940    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-registry-certificates" pod "1e6866c0-b229-4cd8-b977-32d366cced05" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:06:33.691404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.691336    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-trusted-ca" pod "1e6866c0-b229-4cd8-b977-32d366cced05" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:06:33.693118 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.693089    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-image-registry-private-configuration" pod "1e6866c0-b229-4cd8-b977-32d366cced05" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:06:33.693279 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.693255    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-bound-sa-token" pod "1e6866c0-b229-4cd8-b977-32d366cced05" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:06:33.693325 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.693281    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-installation-pull-secrets" pod "1e6866c0-b229-4cd8-b977-32d366cced05" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:06:33.693325 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.693290    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls" pod "1e6866c0-b229-4cd8-b977-32d366cced05" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:06:33.693447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.693426    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-kube-api-access-2dx64" pod "1e6866c0-b229-4cd8-b977-32d366cced05" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05"). InnerVolumeSpecName "kube-api-access-2dx64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:06:33.697904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.697883    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e6866c0-b229-4cd8-b977-32d366cced05-ca-trust-extracted" pod "1e6866c0-b229-4cd8-b977-32d366cced05" (UID: "1e6866c0-b229-4cd8-b977-32d366cced05"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:06:33.792228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.792156    5067 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-registry-tls\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:33.792228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.792183    5067 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-trusted-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:33.792228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.792194    5067 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-image-registry-private-configuration\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:33.792228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.792205    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dx64\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-kube-api-access-2dx64\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:33.792228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.792215    5067 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e6866c0-b229-4cd8-b977-32d366cced05-registry-certificates\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:33.792228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.792225    5067 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e6866c0-b229-4cd8-b977-32d366cced05-installation-pull-secrets\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:33.792228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.792234    5067 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e6866c0-b229-4cd8-b977-32d366cced05-ca-trust-extracted\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:33.792578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:33.792242    5067 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e6866c0-b229-4cd8-b977-32d366cced05-bound-sa-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:34.248968 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:34.248936    5067 generic.go:356] "Generic (PLEG): container finished" podID="1e6866c0-b229-4cd8-b977-32d366cced05" containerID="11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31" exitCode=0
Nov 02 22:06:34.249137 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:34.248980    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56dc494c89-qz46b"
Nov 02 22:06:34.249137 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:34.248993    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56dc494c89-qz46b" event={"ID":"1e6866c0-b229-4cd8-b977-32d366cced05","Type":"ContainerDied","Data":"11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31"}
Nov 02 22:06:34.249137 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:34.249034    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56dc494c89-qz46b" event={"ID":"1e6866c0-b229-4cd8-b977-32d366cced05","Type":"ContainerDied","Data":"8f1fc6ad6e7eff6077f474717820d6a8dadc7a4e93fc852bac325f63d1f288c3"}
Nov 02 22:06:34.249137 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:34.249060    5067 scope.go:122] "RemoveContainer" containerID="11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31"
Nov 02 22:06:34.257375 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:34.257206    5067 scope.go:122] "RemoveContainer" containerID="11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31"
Nov 02 22:06:34.257727 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:06:34.257491    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31\": container with ID starting with 11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31 not found: ID does not exist" containerID="11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31"
Nov 02 22:06:34.257727 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:34.257516    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31"} err="failed to get container status \"11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31\": rpc error: code = NotFound desc = could not find container \"11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31\": container with ID starting with 11aaa6c6f26f4e19f7f6d4782b8e3b75fca11a3c4626e5da1e52a8a77c182e31 not found: ID does not exist"
Nov 02 22:06:34.449785 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:34.449754    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56dc494c89-qz46b"]
Nov 02 22:06:34.496318 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:34.496288    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56dc494c89-qz46b"]
Nov 02 22:06:35.843618 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:35.843586    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e6866c0-b229-4cd8-b977-32d366cced05" path="/var/lib/kubelet/pods/1e6866c0-b229-4cd8-b977-32d366cced05/volumes"
Nov 02 22:06:37.576421 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:06:37.576353    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:06:43.287884 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:43.287847    5067 generic.go:356] "Generic (PLEG): container finished" podID="3465159d-8024-48a2-8408-eb1d1db96a8a" containerID="0b492fb520d1ba13609e8bc8f61b5bfdc4bfaf7f62f1a6db5d5b325d0d87b5d7" exitCode=0
Nov 02 22:06:43.288306 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:43.287920    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-9tsq4" event={"ID":"3465159d-8024-48a2-8408-eb1d1db96a8a","Type":"ContainerDied","Data":"0b492fb520d1ba13609e8bc8f61b5bfdc4bfaf7f62f1a6db5d5b325d0d87b5d7"}
Nov 02 22:06:43.288306 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:43.288260    5067 scope.go:122] "RemoveContainer" containerID="0b492fb520d1ba13609e8bc8f61b5bfdc4bfaf7f62f1a6db5d5b325d0d87b5d7"
Nov 02 22:06:44.269459 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.269418    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-qzwrq-r69mk"]
Nov 02 22:06:44.269680 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.269652    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-qzwrq-r69mk" podUID="04b23efe-ed57-42e2-bf7a-f51bb52e06c0" containerName="insights-gathering" containerID="cri-o://200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b" gracePeriod=30
Nov 02 22:06:44.294016 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.293985    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-9tsq4" event={"ID":"3465159d-8024-48a2-8408-eb1d1db96a8a","Type":"ContainerStarted","Data":"89ea4b71cb28a2b39d3d318bc57c5a29a73a86b89a6e513dcb7ed88adc97d853"}
Nov 02 22:06:44.557210 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.557179    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:06:44.697441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.697403    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-kube-api-access-b45cw\" (UniqueName: \"kubernetes.io/projected/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-kube-api-access-b45cw\") pod \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") "
Nov 02 22:06:44.697609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.697464    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-service-ca-bundle\") pod \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") "
Nov 02 22:06:44.697609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.697502    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-archives-path\") pod \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") "
Nov 02 22:06:44.697609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.697543    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-trusted-ca-bundle\") pod \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\" (UID: \"04b23efe-ed57-42e2-bf7a-f51bb52e06c0\") "
Nov 02 22:06:44.697866 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.697842    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-archives-path" pod "04b23efe-ed57-42e2-bf7a-f51bb52e06c0" (UID: "04b23efe-ed57-42e2-bf7a-f51bb52e06c0"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:06:44.697866 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.697855    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-service-ca-bundle" pod "04b23efe-ed57-42e2-bf7a-f51bb52e06c0" (UID: "04b23efe-ed57-42e2-bf7a-f51bb52e06c0"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:06:44.697992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.697942    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-trusted-ca-bundle" pod "04b23efe-ed57-42e2-bf7a-f51bb52e06c0" (UID: "04b23efe-ed57-42e2-bf7a-f51bb52e06c0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:06:44.699690 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.699668    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-kube-api-access-b45cw" pod "04b23efe-ed57-42e2-bf7a-f51bb52e06c0" (UID: "04b23efe-ed57-42e2-bf7a-f51bb52e06c0"). InnerVolumeSpecName "kube-api-access-b45cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:06:44.798492 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.798411    5067 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-trusted-ca-bundle\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:44.798492 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.798441    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b45cw\" (UniqueName: \"kubernetes.io/projected/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-kube-api-access-b45cw\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:44.798492 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.798453    5067 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-service-ca-bundle\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:44.798492 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:44.798464    5067 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04b23efe-ed57-42e2-bf7a-f51bb52e06c0-archives-path\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:06:45.298992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:45.298956    5067 generic.go:356] "Generic (PLEG): container finished" podID="04b23efe-ed57-42e2-bf7a-f51bb52e06c0" containerID="200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b" exitCode=2
Nov 02 22:06:45.299413 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:45.298998    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-qzwrq-r69mk"
Nov 02 22:06:45.299413 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:45.299036    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-qzwrq-r69mk" event={"ID":"04b23efe-ed57-42e2-bf7a-f51bb52e06c0","Type":"ContainerDied","Data":"200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b"}
Nov 02 22:06:45.299413 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:45.299093    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-qzwrq-r69mk" event={"ID":"04b23efe-ed57-42e2-bf7a-f51bb52e06c0","Type":"ContainerDied","Data":"8a6946e110986354bec312abf97075b91e18efbcaf6a19244df0374b4834364c"}
Nov 02 22:06:45.299413 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:45.299113    5067 scope.go:122] "RemoveContainer" containerID="200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b"
Nov 02 22:06:45.307772 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:45.307750    5067 scope.go:122] "RemoveContainer" containerID="200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b"
Nov 02 22:06:45.308064 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:06:45.308035    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b\": container with ID starting with 200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b not found: ID does not exist" containerID="200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b"
Nov 02 22:06:45.308159 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:45.308077    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b"} err="failed to get container status \"200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b\": rpc error: code = NotFound desc = could not find container \"200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b\": container with ID starting with 200c3e4234883a60d0088ac895f696a1fec827c30ea24b90c1ba0e299a05e00b not found: ID does not exist"
Nov 02 22:06:45.452503 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:45.452471    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-qzwrq-r69mk"]
Nov 02 22:06:45.498656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:45.498623    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-qzwrq-r69mk"]
Nov 02 22:06:45.843901 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:45.843870    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04b23efe-ed57-42e2-bf7a-f51bb52e06c0" path="/var/lib/kubelet/pods/04b23efe-ed57-42e2-bf7a-f51bb52e06c0/volumes"
Nov 02 22:06:53.334226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:53.334140    5067 generic.go:356] "Generic (PLEG): container finished" podID="a2c3d84e-0830-4800-bb72-28bd2b6ca4c5" containerID="2df6df52df6a30c7b856faebf62cc71d5d3d0a3253ae587870dc8a2389bfe913" exitCode=0
Nov 02 22:06:53.334226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:53.334205    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7" event={"ID":"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5","Type":"ContainerDied","Data":"2df6df52df6a30c7b856faebf62cc71d5d3d0a3253ae587870dc8a2389bfe913"}
Nov 02 22:06:53.334698 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:53.334544    5067 scope.go:122] "RemoveContainer" containerID="2df6df52df6a30c7b856faebf62cc71d5d3d0a3253ae587870dc8a2389bfe913"
Nov 02 22:06:54.339985 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:06:54.339950    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mbcx7" event={"ID":"a2c3d84e-0830-4800-bb72-28bd2b6ca4c5","Type":"ContainerStarted","Data":"542ba115c982a315a753a63d2467d83b127dc73164316a1dd48268237ed723c9"}
Nov 02 22:07:07.580814 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:07:07.580774    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:07:37.575782 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:07:37.575745    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:08:07.576696 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:08:07.576657    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:08:37.574464 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:08:37.574421    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:09:06.870715 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.870677    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"]
Nov 02 22:09:06.871349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.871136    5067 cpu_manager.go:414] "containerMap: removing container" podUID="04b23efe-ed57-42e2-bf7a-f51bb52e06c0" containerName="insights-gathering"
Nov 02 22:09:06.871349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.871153    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04b23efe-ed57-42e2-bf7a-f51bb52e06c0" containerName="insights-gathering"
Nov 02 22:09:06.871349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.871169    5067 cpu_manager.go:414] "containerMap: removing container" podUID="a8b9eeac-3e3f-4612-a696-2ede201d83ac" containerName="registry"
Nov 02 22:09:06.871349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.871177    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8b9eeac-3e3f-4612-a696-2ede201d83ac" containerName="registry"
Nov 02 22:09:06.871349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.871221    5067 cpu_manager.go:414] "containerMap: removing container" podUID="1e6866c0-b229-4cd8-b977-32d366cced05" containerName="registry"
Nov 02 22:09:06.871349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.871230    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e6866c0-b229-4cd8-b977-32d366cced05" containerName="registry"
Nov 02 22:09:06.871349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.871312    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="04b23efe-ed57-42e2-bf7a-f51bb52e06c0" containerName="insights-gathering"
Nov 02 22:09:06.871349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.871322    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8b9eeac-3e3f-4612-a696-2ede201d83ac" containerName="registry"
Nov 02 22:09:06.871349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.871338    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e6866c0-b229-4cd8-b977-32d366cced05" containerName="registry"
Nov 02 22:09:06.873806 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.873784    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:06.922909 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.922881    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 02 22:09:06.943524 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.943498    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-q825d\""
Nov 02 22:09:06.968848 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.968821    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:06.968949 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.968854    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:06.968991 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.968942    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:06.969025 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.969010    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:06.969058 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.969032    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:06.969097 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.969067    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:06.969285 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.969269    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:06.969329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.969299    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:06.969401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.969314    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgcvj\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-kube-api-access-rgcvj\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:06.971681 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:06.971657    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"]
Nov 02 22:09:07.070762 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.070724    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.070944 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.070768    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.070944 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.070892    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgcvj\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-kube-api-access-rgcvj\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.070944 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.070925    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.071131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.070955    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.071131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.070998    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.071131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.071083    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.071262 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.071134    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.071262 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.071188    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.071483 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.071457    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.071552 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.071494    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.071552 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.071538    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.071699 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.071683    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.072025 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.072004    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.073041 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.073012    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.073164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.073146    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.128805 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.128743    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.128924 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.128869    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgcvj\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-kube-api-access-rgcvj\") pod \"maas-default-gateway-openshift-default-76f4876667-7sfm7\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.171917 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.171886    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"]
Nov 02 22:09:07.174449 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.174434    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.184731 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.184711    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:07.235486 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.234776    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"]
Nov 02 22:09:07.273021 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.272987    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.273188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.273029    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttjdh\" (UniqueName: \"kubernetes.io/projected/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-kube-api-access-ttjdh\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.273188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.273094    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.273188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.273119    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.273188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.273137    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.273188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.273155    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.273500 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.273265    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.273500 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.273294    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.273500 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.273325    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.298747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.298716    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-qsvd5\""
Nov 02 22:09:07.373755 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.373714    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.373894 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.373763    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.373894 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.373792    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.373894 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.373838    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.373894 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.373867    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.374124 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.373909    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.374124 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.373953    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.374124 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.373993    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttjdh\" (UniqueName: \"kubernetes.io/projected/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-kube-api-access-ttjdh\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.374124 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.374031    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.374309 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.374155    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.374309 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.374221    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.374457 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.374434    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.374518 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.374466    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.374650 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.374628    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.376041 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.376018    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.376659 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.376643    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.431950 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.431879    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.431950 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.431944    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttjdh\" (UniqueName: \"kubernetes.io/projected/49754aeb-a62e-4c38-bfb0-75dbc9f12c40-kube-api-access-ttjdh\") pod \"openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2\" (UID: \"49754aeb-a62e-4c38-bfb0-75dbc9f12c40\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.485624 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.485590    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:07.578148 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:09:07.578108    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:09:07.608408 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.608233    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"]
Nov 02 22:09:07.611049 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:09:07.611015    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ddee653_e8f0_42a6_9c14_3361ccc6dc97.slice/crio-dea50c39ec90f665443547f22454a5df1281725d09aaaa9f10d2323d0acaf176 WatchSource:0}: Error finding container dea50c39ec90f665443547f22454a5df1281725d09aaaa9f10d2323d0acaf176: Status 404 returned error can't find the container with id dea50c39ec90f665443547f22454a5df1281725d09aaaa9f10d2323d0acaf176
Nov 02 22:09:07.857856 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.857655    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"]
Nov 02 22:09:07.860510 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:09:07.860487    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49754aeb_a62e_4c38_bfb0_75dbc9f12c40.slice/crio-89861d48922bf309fafb937e466a7bd3bd0115076081b1c20c6f59a863cbdb3a WatchSource:0}: Error finding container 89861d48922bf309fafb937e466a7bd3bd0115076081b1c20c6f59a863cbdb3a: Status 404 returned error can't find the container with id 89861d48922bf309fafb937e466a7bd3bd0115076081b1c20c6f59a863cbdb3a
Nov 02 22:09:07.869266 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.869236    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2" event={"ID":"49754aeb-a62e-4c38-bfb0-75dbc9f12c40","Type":"ContainerStarted","Data":"89861d48922bf309fafb937e466a7bd3bd0115076081b1c20c6f59a863cbdb3a"}
Nov 02 22:09:07.870261 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:07.870241    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" event={"ID":"0ddee653-e8f0-42a6-9c14-3361ccc6dc97","Type":"ContainerStarted","Data":"dea50c39ec90f665443547f22454a5df1281725d09aaaa9f10d2323d0acaf176"}
Nov 02 22:09:12.035699 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.035655    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:09:12.036057 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.035735    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:09:12.036057 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.035769    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:09:12.533035 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.533002    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"]
Nov 02 22:09:12.536385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.536345    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.568029 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.567999    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:09:12.568113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.568065    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:09:12.568113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.568094    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:09:12.622610 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.622580    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx5xj\" (UniqueName: \"kubernetes.io/projected/b1fca734-caf6-4ce9-9055-b1701e64e81b-kube-api-access-xx5xj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.622726 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.622621    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.622771 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.622749    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.633821 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.633784    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-vh5f8\""
Nov 02 22:09:12.634020 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.633993    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 02 22:09:12.634153 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.634086    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 02 22:09:12.636016 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.635992    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"]
Nov 02 22:09:12.723175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.723142    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.723348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.723194    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx5xj\" (UniqueName: \"kubernetes.io/projected/b1fca734-caf6-4ce9-9055-b1701e64e81b-kube-api-access-xx5xj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.723348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.723309    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.723627 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.723608    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.723697 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.723634    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.781347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.781312    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx5xj\" (UniqueName: \"kubernetes.io/projected/b1fca734-caf6-4ce9-9055-b1701e64e81b-kube-api-access-xx5xj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.850841 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.850749    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:12.905207 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.905072    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" event={"ID":"0ddee653-e8f0-42a6-9c14-3361ccc6dc97","Type":"ContainerStarted","Data":"f3432dddba01ac5a6cabae65d3db995397765409a6110439e0c85c9336e0ed9e"}
Nov 02 22:09:12.907606 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:12.907577    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2" event={"ID":"49754aeb-a62e-4c38-bfb0-75dbc9f12c40","Type":"ContainerStarted","Data":"bd1f32aa35d3a0ff13b0e10b2df4bc4701804bd6a3498e4bd2add7f86996bf1f"}
Nov 02 22:09:13.020724 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.020663    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" podStartSLOduration=2.066322134 podStartE2EDuration="7.020648074s" totalImagesPullingTime="4.95432594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:13.019631715 +0000 UTC m=+225.303441393" watchObservedRunningTime="2026-11-02 22:09:13.020648074 +0000 UTC m=+225.304457761"
Nov 02 22:09:13.136353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.136221    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2" podStartSLOduration=1.9638185849999998 podStartE2EDuration="6.136200687s" totalImagesPullingTime="4.172382102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:09:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:09:13.134258176 +0000 UTC m=+225.418067883" watchObservedRunningTime="2026-11-02 22:09:13.136200687 +0000 UTC m=+225.420010375"
Nov 02 22:09:13.185715 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.185676    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:13.190276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.190248    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:13.267883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.267838    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"]
Nov 02 22:09:13.272068 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:09:13.272035    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1fca734_caf6_4ce9_9055_b1701e64e81b.slice/crio-92ae389f8a38ec20c641d58d175d03556d6d402ff4adb5fdfda7524c3ee65e95 WatchSource:0}: Error finding container 92ae389f8a38ec20c641d58d175d03556d6d402ff4adb5fdfda7524c3ee65e95: Status 404 returned error can't find the container with id 92ae389f8a38ec20c641d58d175d03556d6d402ff4adb5fdfda7524c3ee65e95
Nov 02 22:09:13.486095 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.486068    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:13.491008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.490983    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:13.913258 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.913154    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7" event={"ID":"b1fca734-caf6-4ce9-9055-b1701e64e81b","Type":"ContainerStarted","Data":"92ae389f8a38ec20c641d58d175d03556d6d402ff4adb5fdfda7524c3ee65e95"}
Nov 02 22:09:13.913540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.913521    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:13.913622 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.913571    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:13.914463 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.914442    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2"
Nov 02 22:09:13.914568 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:13.914500    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:09:19.944049 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:19.943956    5067 generic.go:356] "Generic (PLEG): container finished" podID="b1fca734-caf6-4ce9-9055-b1701e64e81b" containerID="e2b495fed7f40589209b9321d77056254e69190f9e15752dc8e4a3c1a36bf387" exitCode=0
Nov 02 22:09:19.944498 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:19.944043    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7" event={"ID":"b1fca734-caf6-4ce9-9055-b1701e64e81b","Type":"ContainerDied","Data":"e2b495fed7f40589209b9321d77056254e69190f9e15752dc8e4a3c1a36bf387"}
Nov 02 22:09:21.953893 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:21.953861    5067 generic.go:356] "Generic (PLEG): container finished" podID="b1fca734-caf6-4ce9-9055-b1701e64e81b" containerID="234cfa7871a2087a0d65a2fdeefc5949771bd223831072b503644595b4f8541b" exitCode=0
Nov 02 22:09:21.954331 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:21.953952    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7" event={"ID":"b1fca734-caf6-4ce9-9055-b1701e64e81b","Type":"ContainerDied","Data":"234cfa7871a2087a0d65a2fdeefc5949771bd223831072b503644595b4f8541b"}
Nov 02 22:09:28.991511 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:28.991476    5067 generic.go:356] "Generic (PLEG): container finished" podID="b1fca734-caf6-4ce9-9055-b1701e64e81b" containerID="895f0098c6cd15040a8afba75967ad9f8edd28fdf04a5342e95ad09d6bf64957" exitCode=0
Nov 02 22:09:28.991914 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:28.991556    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7" event={"ID":"b1fca734-caf6-4ce9-9055-b1701e64e81b","Type":"ContainerDied","Data":"895f0098c6cd15040a8afba75967ad9f8edd28fdf04a5342e95ad09d6bf64957"}
Nov 02 22:09:30.123951 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:30.123924    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:30.281569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:30.281495    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-bundle\") pod \"b1fca734-caf6-4ce9-9055-b1701e64e81b\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") "
Nov 02 22:09:30.281569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:30.281532    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-util\" (UniqueName: \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-util\") pod \"b1fca734-caf6-4ce9-9055-b1701e64e81b\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") "
Nov 02 22:09:30.281814 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:30.281572    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1fca734-caf6-4ce9-9055-b1701e64e81b-kube-api-access-xx5xj\" (UniqueName: \"kubernetes.io/projected/b1fca734-caf6-4ce9-9055-b1701e64e81b-kube-api-access-xx5xj\") pod \"b1fca734-caf6-4ce9-9055-b1701e64e81b\" (UID: \"b1fca734-caf6-4ce9-9055-b1701e64e81b\") "
Nov 02 22:09:30.282011 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:30.281986    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-bundle" pod "b1fca734-caf6-4ce9-9055-b1701e64e81b" (UID: "b1fca734-caf6-4ce9-9055-b1701e64e81b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:30.283916 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:30.283892    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1fca734-caf6-4ce9-9055-b1701e64e81b-kube-api-access-xx5xj" pod "b1fca734-caf6-4ce9-9055-b1701e64e81b" (UID: "b1fca734-caf6-4ce9-9055-b1701e64e81b"). InnerVolumeSpecName "kube-api-access-xx5xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:09:30.286311 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:30.286285    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-util" pod "b1fca734-caf6-4ce9-9055-b1701e64e81b" (UID: "b1fca734-caf6-4ce9-9055-b1701e64e81b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:09:30.383196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:30.383162    5067 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-bundle\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:09:30.383196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:30.383190    5067 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1fca734-caf6-4ce9-9055-b1701e64e81b-util\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:09:30.383196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:30.383200    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx5xj\" (UniqueName: \"kubernetes.io/projected/b1fca734-caf6-4ce9-9055-b1701e64e81b-kube-api-access-xx5xj\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:09:31.005989 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:31.005953    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7" event={"ID":"b1fca734-caf6-4ce9-9055-b1701e64e81b","Type":"ContainerDied","Data":"92ae389f8a38ec20c641d58d175d03556d6d402ff4adb5fdfda7524c3ee65e95"}
Nov 02 22:09:31.006167 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:31.006029    5067 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92ae389f8a38ec20c641d58d175d03556d6d402ff4adb5fdfda7524c3ee65e95"
Nov 02 22:09:31.006167 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:09:31.005963    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxjbf7"
Nov 02 22:09:37.574423 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:09:37.574357    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:10:06.380765 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.380722    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976"]
Nov 02 22:10:06.382570 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.381239    5067 cpu_manager.go:414] "containerMap: removing container" podUID="b1fca734-caf6-4ce9-9055-b1701e64e81b" containerName="extract"
Nov 02 22:10:06.382570 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.381255    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1fca734-caf6-4ce9-9055-b1701e64e81b" containerName="extract"
Nov 02 22:10:06.382570 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.381279    5067 cpu_manager.go:414] "containerMap: removing container" podUID="b1fca734-caf6-4ce9-9055-b1701e64e81b" containerName="pull"
Nov 02 22:10:06.382570 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.381284    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1fca734-caf6-4ce9-9055-b1701e64e81b" containerName="pull"
Nov 02 22:10:06.382570 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.381296    5067 cpu_manager.go:414] "containerMap: removing container" podUID="b1fca734-caf6-4ce9-9055-b1701e64e81b" containerName="util"
Nov 02 22:10:06.382570 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.381302    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1fca734-caf6-4ce9-9055-b1701e64e81b" containerName="util"
Nov 02 22:10:06.382570 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.381359    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1fca734-caf6-4ce9-9055-b1701e64e81b" containerName="extract"
Nov 02 22:10:06.383433 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.383414    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976"
Nov 02 22:10:06.452278 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.452246    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 02 22:10:06.487745 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.487710    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Nov 02 22:10:06.487745 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.487721    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 02 22:10:06.489101 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.489079    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976"]
Nov 02 22:10:06.490210 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.490188    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96rnl\" (UniqueName: \"kubernetes.io/projected/de500486-6d58-48e7-80e3-81b3ff9cbbef-kube-api-access-96rnl\") pod \"dns-operator-controller-manager-8557cbdb84-hd976\" (UID: \"de500486-6d58-48e7-80e3-81b3ff9cbbef\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976"
Nov 02 22:10:06.534167 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.534136    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-q2tnn\""
Nov 02 22:10:06.591053 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.591020    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96rnl\" (UniqueName: \"kubernetes.io/projected/de500486-6d58-48e7-80e3-81b3ff9cbbef-kube-api-access-96rnl\") pod \"dns-operator-controller-manager-8557cbdb84-hd976\" (UID: \"de500486-6d58-48e7-80e3-81b3ff9cbbef\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976"
Nov 02 22:10:06.648903 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.648833    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96rnl\" (UniqueName: \"kubernetes.io/projected/de500486-6d58-48e7-80e3-81b3ff9cbbef-kube-api-access-96rnl\") pod \"dns-operator-controller-manager-8557cbdb84-hd976\" (UID: \"de500486-6d58-48e7-80e3-81b3ff9cbbef\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976"
Nov 02 22:10:06.695757 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:06.695724    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976"
Nov 02 22:10:07.067135 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:07.067101    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976"]
Nov 02 22:10:07.072101 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:10:07.072072    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde500486_6d58_48e7_80e3_81b3ff9cbbef.slice/crio-6a6905811163dfe704be95838522dbf6c03ddb4288bc2350ababb52ce8f6e543 WatchSource:0}: Error finding container 6a6905811163dfe704be95838522dbf6c03ddb4288bc2350ababb52ce8f6e543: Status 404 returned error can't find the container with id 6a6905811163dfe704be95838522dbf6c03ddb4288bc2350ababb52ce8f6e543
Nov 02 22:10:07.156928 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:07.156889    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976" event={"ID":"de500486-6d58-48e7-80e3-81b3ff9cbbef","Type":"ContainerStarted","Data":"6a6905811163dfe704be95838522dbf6c03ddb4288bc2350ababb52ce8f6e543"}
Nov 02 22:10:07.577066 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:10:07.577023    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:10:09.584814 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.584774    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"]
Nov 02 22:10:09.587898 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.587874    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.639915 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.639883    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 02 22:10:09.639915 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.639903    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 02 22:10:09.640126 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.639937    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 02 22:10:09.640126 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.640016    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 02 22:10:09.640126 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.640087    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-kldsf\""
Nov 02 22:10:09.640250 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.640166    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 02 22:10:09.705629 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.705595    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"]
Nov 02 22:10:09.720938 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.720898    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-metrics-cert\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.721111 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.720946    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-webhook-cert\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.721111 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.721019    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.721227 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.721128    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-tmp\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.721227 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.721166    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-manifests\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.721312 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.721255    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7b92\" (UniqueName: \"kubernetes.io/projected/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-kube-api-access-m7b92\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.821980 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.821947    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7b92\" (UniqueName: \"kubernetes.io/projected/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-kube-api-access-m7b92\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.822176 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.822010    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-metrics-cert\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.822176 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.822026    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-webhook-cert\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.822176 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.822111    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.822176 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.822172    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-tmp\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.822426 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.822203    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-manifests\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.822638 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.822611    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-manifests\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.824783 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.824744    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.824891 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.824875    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-tmp\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.824971 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.824953    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-metrics-cert\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.825141 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.825127    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-webhook-cert\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.883034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.882951    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7b92\" (UniqueName: \"kubernetes.io/projected/1dc7fb9e-78e6-4314-94d3-a5e2d169db12-kube-api-access-m7b92\") pod \"opendatahub-operator-controller-manager-d99fd484b-fkdbz\" (UID: \"1dc7fb9e-78e6-4314-94d3-a5e2d169db12\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:09.904632 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:09.904606    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:10.464744 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:10.464705    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"]
Nov 02 22:10:10.469541 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:10:10.469505    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc7fb9e_78e6_4314_94d3_a5e2d169db12.slice/crio-a1c7bcc33fa42f49ff9a5ae4d4ee945a0b27f10b933a8956b97ab7c705952701 WatchSource:0}: Error finding container a1c7bcc33fa42f49ff9a5ae4d4ee945a0b27f10b933a8956b97ab7c705952701: Status 404 returned error can't find the container with id a1c7bcc33fa42f49ff9a5ae4d4ee945a0b27f10b933a8956b97ab7c705952701
Nov 02 22:10:11.185213 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:11.185105    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976" event={"ID":"de500486-6d58-48e7-80e3-81b3ff9cbbef","Type":"ContainerStarted","Data":"b49739a0c7c9d4beb5252dc2e6d72ad0fe8d751b43625dc3485c78f6841db6ec"}
Nov 02 22:10:11.185213 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:11.185171    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976"
Nov 02 22:10:11.186502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:11.186479    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz" event={"ID":"1dc7fb9e-78e6-4314-94d3-a5e2d169db12","Type":"ContainerStarted","Data":"a1c7bcc33fa42f49ff9a5ae4d4ee945a0b27f10b933a8956b97ab7c705952701"}
Nov 02 22:10:11.307242 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:11.307180    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976" podStartSLOduration=1.4867622360000001 podStartE2EDuration="5.307165913s" totalImagesPullingTime="3.820403677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:10:11.306648895 +0000 UTC m=+283.590458582" watchObservedRunningTime="2026-11-02 22:10:11.307165913 +0000 UTC m=+283.590975600"
Nov 02 22:10:14.204172 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:14.204138    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz" event={"ID":"1dc7fb9e-78e6-4314-94d3-a5e2d169db12","Type":"ContainerStarted","Data":"b6b06c2c08708bf5bb1db33487f378423538f25caddbee814a77163ecd747d86"}
Nov 02 22:10:14.958857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:14.958761    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"]
Nov 02 22:10:14.961619 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:14.961593    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:10:15.012813 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.012774    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rtphq\""
Nov 02 22:10:15.023678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.023648    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"]
Nov 02 22:10:15.072633 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.072599    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/524851bd-08e5-4413-8e20-4e893631f23e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-mrt2k\" (UID: \"524851bd-08e5-4413-8e20-4e893631f23e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:10:15.072785 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.072645    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d64g\" (UniqueName: \"kubernetes.io/projected/524851bd-08e5-4413-8e20-4e893631f23e-kube-api-access-8d64g\") pod \"kuadrant-operator-controller-manager-678d87766f-mrt2k\" (UID: \"524851bd-08e5-4413-8e20-4e893631f23e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:10:15.173341 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.173301    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/524851bd-08e5-4413-8e20-4e893631f23e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-mrt2k\" (UID: \"524851bd-08e5-4413-8e20-4e893631f23e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:10:15.173528 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.173354    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d64g\" (UniqueName: \"kubernetes.io/projected/524851bd-08e5-4413-8e20-4e893631f23e-kube-api-access-8d64g\") pod \"kuadrant-operator-controller-manager-678d87766f-mrt2k\" (UID: \"524851bd-08e5-4413-8e20-4e893631f23e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:10:15.173741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.173720    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/524851bd-08e5-4413-8e20-4e893631f23e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-mrt2k\" (UID: \"524851bd-08e5-4413-8e20-4e893631f23e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:10:15.209626 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.209542    5067 generic.go:356] "Generic (PLEG): container finished" podID="1dc7fb9e-78e6-4314-94d3-a5e2d169db12" containerID="b6b06c2c08708bf5bb1db33487f378423538f25caddbee814a77163ecd747d86" exitCode=0
Nov 02 22:10:15.210008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.209629    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz" event={"ID":"1dc7fb9e-78e6-4314-94d3-a5e2d169db12","Type":"ContainerDied","Data":"b6b06c2c08708bf5bb1db33487f378423538f25caddbee814a77163ecd747d86"}
Nov 02 22:10:15.232195 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.232168    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d64g\" (UniqueName: \"kubernetes.io/projected/524851bd-08e5-4413-8e20-4e893631f23e-kube-api-access-8d64g\") pod \"kuadrant-operator-controller-manager-678d87766f-mrt2k\" (UID: \"524851bd-08e5-4413-8e20-4e893631f23e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:10:15.273713 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.273680    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:10:15.704898 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:15.704867    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"]
Nov 02 22:10:15.708679 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:10:15.708645    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod524851bd_08e5_4413_8e20_4e893631f23e.slice/crio-1602db491bc7a44ab5f895d09e0f166d8310fe75a4bb8b8d03ad8da62808bb70 WatchSource:0}: Error finding container 1602db491bc7a44ab5f895d09e0f166d8310fe75a4bb8b8d03ad8da62808bb70: Status 404 returned error can't find the container with id 1602db491bc7a44ab5f895d09e0f166d8310fe75a4bb8b8d03ad8da62808bb70
Nov 02 22:10:16.222245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:16.222207    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k" event={"ID":"524851bd-08e5-4413-8e20-4e893631f23e","Type":"ContainerStarted","Data":"1602db491bc7a44ab5f895d09e0f166d8310fe75a4bb8b8d03ad8da62808bb70"}
Nov 02 22:10:16.224868 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:16.224843    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz" event={"ID":"1dc7fb9e-78e6-4314-94d3-a5e2d169db12","Type":"ContainerStarted","Data":"ba30fc0d966fb03a2c7f0a13c40d68188154dce768ace2fdc5d3b01f77bb1fcb"}
Nov 02 22:10:16.225006 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:16.224972    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:16.367017 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:16.366961    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz" podStartSLOduration=2.43456301 podStartE2EDuration="7.366944306s" totalImagesPullingTime="4.488088326s" totalInitContainerRuntime="444.29297ms" isStatefulPod=false podCreationTimestamp="2026-11-02 22:10:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:10:16.366063705 +0000 UTC m=+288.649873408" watchObservedRunningTime="2026-11-02 22:10:16.366944306 +0000 UTC m=+288.650754048"
Nov 02 22:10:21.250556 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:21.250510    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k" event={"ID":"524851bd-08e5-4413-8e20-4e893631f23e","Type":"ContainerStarted","Data":"038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4"}
Nov 02 22:10:21.251021 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:21.250615    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:10:21.456882 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:21.456778    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k" podStartSLOduration=2.136686431 podStartE2EDuration="7.456761483s" totalImagesPullingTime="5.320075052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:10:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:10:21.455953069 +0000 UTC m=+293.739762756" watchObservedRunningTime="2026-11-02 22:10:21.456761483 +0000 UTC m=+293.740571169"
Nov 02 22:10:22.195446 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:22.195417    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-hd976"
Nov 02 22:10:27.231337 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:27.231307    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-d99fd484b-fkdbz"
Nov 02 22:10:27.864965 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:27.864929    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:10:27.865757 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:27.865732    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:10:32.256708 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:32.256676    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:10:37.575127 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:10:37.575089    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:10:43.362946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.362911    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"]
Nov 02 22:10:43.410518 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.410490    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.421143 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.421109    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"]
Nov 02 22:10:43.517558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.517521    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51-tmp\") pod \"kube-auth-proxy-6f985b8d7-ph8cc\" (UID: \"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51\") " pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.517745 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.517575    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f68cj\" (UniqueName: \"kubernetes.io/projected/9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51-kube-api-access-f68cj\") pod \"kube-auth-proxy-6f985b8d7-ph8cc\" (UID: \"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51\") " pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.517745 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.517603    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51-tls-certs\") pod \"kube-auth-proxy-6f985b8d7-ph8cc\" (UID: \"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51\") " pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.531886 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.531844    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 02 22:10:43.532044 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.531912    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-84m9x\""
Nov 02 22:10:43.569458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.569428    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 02 22:10:43.619220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.619131    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51-tmp\") pod \"kube-auth-proxy-6f985b8d7-ph8cc\" (UID: \"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51\") " pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.619220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.619192    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f68cj\" (UniqueName: \"kubernetes.io/projected/9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51-kube-api-access-f68cj\") pod \"kube-auth-proxy-6f985b8d7-ph8cc\" (UID: \"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51\") " pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.619512 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.619227    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51-tls-certs\") pod \"kube-auth-proxy-6f985b8d7-ph8cc\" (UID: \"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51\") " pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.621496 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.621469    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51-tmp\") pod \"kube-auth-proxy-6f985b8d7-ph8cc\" (UID: \"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51\") " pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.621649 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.621627    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51-tls-certs\") pod \"kube-auth-proxy-6f985b8d7-ph8cc\" (UID: \"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51\") " pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.678550 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.678520    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f68cj\" (UniqueName: \"kubernetes.io/projected/9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51-kube-api-access-f68cj\") pod \"kube-auth-proxy-6f985b8d7-ph8cc\" (UID: \"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51\") " pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.731806 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.731775    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"
Nov 02 22:10:43.828023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.826705    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"]
Nov 02 22:10:43.850976 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.850948    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:43.887964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.887882    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"]
Nov 02 22:10:43.900048 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.900018    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-w2q4f\""
Nov 02 22:10:43.923400 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.923354    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:43.923582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.923460    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-token\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:43.923582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.923494    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:43.923582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.923572    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-data\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:43.923745 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.923610    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:43.923745 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.923638    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:43.923845 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.923742    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:43.923845 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.923762    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:43.923845 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:43.923788    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5glv4\" (UniqueName: \"kubernetes.io/projected/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-kube-api-access-5glv4\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.024964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.024932    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.025129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.024979    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-token\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.025129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.025008    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.025129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.025050    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-data\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.025129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.025096    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.025129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.025122    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.025347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.025181    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.025347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.025204    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.025347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.025242    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5glv4\" (UniqueName: \"kubernetes.io/projected/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-kube-api-access-5glv4\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.025923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.025891    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-data\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.026076 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.025920    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.026076 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.025986    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.026076 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.026031    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.026257 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.026173    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.027782 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.027759    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.027883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.027871    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.085497 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.085462    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-istio-token\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.085640 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.085587    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5glv4\" (UniqueName: \"kubernetes.io/projected/ec35ca20-d658-4f8a-8725-4d5cd3b5ad87-kube-api-access-5glv4\") pod \"data-science-gateway-data-science-gateway-class-67c76445c-rm6gt\" (UID: \"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.134826 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.134789    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc"]
Nov 02 22:10:44.138180 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:10:44.138113    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3fdd6f_e83a_47a3_8d54_576cc6c2ba51.slice/crio-6cf60089a29ff91a1bfb5535d87613431940db3dac01818390dce539ec830ab2 WatchSource:0}: Error finding container 6cf60089a29ff91a1bfb5535d87613431940db3dac01818390dce539ec830ab2: Status 404 returned error can't find the container with id 6cf60089a29ff91a1bfb5535d87613431940db3dac01818390dce539ec830ab2
Nov 02 22:10:44.140063 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.140046    5067 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:10:44.190123 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.190086    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:44.352677 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.352640    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc" event={"ID":"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51","Type":"ContainerStarted","Data":"6cf60089a29ff91a1bfb5535d87613431940db3dac01818390dce539ec830ab2"}
Nov 02 22:10:44.636213 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.636178    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"]
Nov 02 22:10:44.639291 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:10:44.639261    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec35ca20_d658_4f8a_8725_4d5cd3b5ad87.slice/crio-f6f5f33d0ec218af7ff60d0a409e9cb42c6847302827834eedafeb97f2b01ea0 WatchSource:0}: Error finding container f6f5f33d0ec218af7ff60d0a409e9cb42c6847302827834eedafeb97f2b01ea0: Status 404 returned error can't find the container with id f6f5f33d0ec218af7ff60d0a409e9cb42c6847302827834eedafeb97f2b01ea0
Nov 02 22:10:44.642032 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.641997    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:10:44.642147 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.642078    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:10:44.642147 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:44.642121    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:10:45.361934 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:45.361894    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt" event={"ID":"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87","Type":"ContainerStarted","Data":"b2ba36d311a80eaa830a5565092b3de9df4f7bc6fc23832d08bb73b929fb171d"}
Nov 02 22:10:45.362111 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:45.361960    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt" event={"ID":"ec35ca20-d658-4f8a-8725-4d5cd3b5ad87","Type":"ContainerStarted","Data":"f6f5f33d0ec218af7ff60d0a409e9cb42c6847302827834eedafeb97f2b01ea0"}
Nov 02 22:10:45.478909 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:45.478840    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt" podStartSLOduration=2.478818723 podStartE2EDuration="2.478818723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:10:45.4775851 +0000 UTC m=+317.761394812" watchObservedRunningTime="2026-11-02 22:10:45.478818723 +0000 UTC m=+317.762628411"
Nov 02 22:10:46.190877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:46.190843    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:46.196227 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:46.196198    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:46.367068 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:46.367034    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:46.368038 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:46.368012    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-67c76445c-rm6gt"
Nov 02 22:10:48.378256 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:48.378215    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc" event={"ID":"9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51","Type":"ContainerStarted","Data":"eee080f5aade339b79fef16890f51a0f022fe4cdbca22862f4605389d81ea89e"}
Nov 02 22:10:48.518484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:48.518434    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6f985b8d7-ph8cc" podStartSLOduration=1.93230951 podStartE2EDuration="5.518418454s" totalImagesPullingTime="3.586108944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:10:48.517025189 +0000 UTC m=+320.800834900" watchObservedRunningTime="2026-11-02 22:10:48.518418454 +0000 UTC m=+320.802228141"
Nov 02 22:10:49.943951 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:49.943907    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2d9jk"]
Nov 02 22:10:49.946514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:49.946490    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk"
Nov 02 22:10:49.999221 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:49.999169    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-88hkt\""
Nov 02 22:10:50.001120 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:50.001089    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2d9jk"]
Nov 02 22:10:50.084055 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:50.084025    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw64x\" (UniqueName: \"kubernetes.io/projected/3766427f-9f22-4e73-afa7-5a0b7faa237b-kube-api-access-xw64x\") pod \"authorino-56c6c9ddd5-2d9jk\" (UID: \"3766427f-9f22-4e73-afa7-5a0b7faa237b\") " pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk"
Nov 02 22:10:50.184864 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:50.184824    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw64x\" (UniqueName: \"kubernetes.io/projected/3766427f-9f22-4e73-afa7-5a0b7faa237b-kube-api-access-xw64x\") pod \"authorino-56c6c9ddd5-2d9jk\" (UID: \"3766427f-9f22-4e73-afa7-5a0b7faa237b\") " pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk"
Nov 02 22:10:50.242671 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:50.242635    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw64x\" (UniqueName: \"kubernetes.io/projected/3766427f-9f22-4e73-afa7-5a0b7faa237b-kube-api-access-xw64x\") pod \"authorino-56c6c9ddd5-2d9jk\" (UID: \"3766427f-9f22-4e73-afa7-5a0b7faa237b\") " pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk"
Nov 02 22:10:50.274307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:50.274275    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk"
Nov 02 22:10:50.638311 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:50.638229    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2d9jk"]
Nov 02 22:10:50.641516 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:10:50.641487    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3766427f_9f22_4e73_afa7_5a0b7faa237b.slice/crio-2ac4d0ab56acae9e785ab4bb1bc772ad5f1c3df73c5d1c2da1c068b2fe070bbd WatchSource:0}: Error finding container 2ac4d0ab56acae9e785ab4bb1bc772ad5f1c3df73c5d1c2da1c068b2fe070bbd: Status 404 returned error can't find the container with id 2ac4d0ab56acae9e785ab4bb1bc772ad5f1c3df73c5d1c2da1c068b2fe070bbd
Nov 02 22:10:51.393007 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:51.392967    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk" event={"ID":"3766427f-9f22-4e73-afa7-5a0b7faa237b","Type":"ContainerStarted","Data":"2ac4d0ab56acae9e785ab4bb1bc772ad5f1c3df73c5d1c2da1c068b2fe070bbd"}
Nov 02 22:10:53.403601 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:53.403506    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk" event={"ID":"3766427f-9f22-4e73-afa7-5a0b7faa237b","Type":"ContainerStarted","Data":"0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a"}
Nov 02 22:10:53.511810 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:10:53.511746    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk" podStartSLOduration=2.023810027 podStartE2EDuration="4.511729128s" totalImagesPullingTime="2.487919101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:10:53.510053955 +0000 UTC m=+325.793863645" watchObservedRunningTime="2026-11-02 22:10:53.511729128 +0000 UTC m=+325.795538815"
Nov 02 22:11:07.576273 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:11:07.576226    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:11:31.625731 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.625691    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-p7mck"]
Nov 02 22:11:31.631144 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.631120    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:31.687703 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.687453    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 02 22:11:31.687877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.687569    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 02 22:11:31.726629 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.726595    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jsb8q\""
Nov 02 22:11:31.726833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.726648    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 22:11:31.726833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.726660    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 02 22:11:31.726833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.726693    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 02 22:11:31.726992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.726931    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-metrics-certs\") pod \"maas-controller-579cf7c6d-p7mck\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") " pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:31.726992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.726980    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlcsj\" (UniqueName: \"kubernetes.io/projected/6e30cf89-5b1d-4c36-a049-4058f27451a5-kube-api-access-nlcsj\") pod \"maas-controller-579cf7c6d-p7mck\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") " pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:31.727096 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.727067    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-webhook-cert\") pod \"maas-controller-579cf7c6d-p7mck\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") " pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:31.728753 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.728723    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-p7mck"]
Nov 02 22:11:31.828308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.828279    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlcsj\" (UniqueName: \"kubernetes.io/projected/6e30cf89-5b1d-4c36-a049-4058f27451a5-kube-api-access-nlcsj\") pod \"maas-controller-579cf7c6d-p7mck\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") " pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:31.828502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.828324    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-webhook-cert\") pod \"maas-controller-579cf7c6d-p7mck\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") " pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:31.828502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.828395    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-metrics-certs\") pod \"maas-controller-579cf7c6d-p7mck\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") " pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:31.830898 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.830866    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-webhook-cert\") pod \"maas-controller-579cf7c6d-p7mck\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") " pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:31.831025 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.830896    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-metrics-certs\") pod \"maas-controller-579cf7c6d-p7mck\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") " pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:31.886490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.886420    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlcsj\" (UniqueName: \"kubernetes.io/projected/6e30cf89-5b1d-4c36-a049-4058f27451a5-kube-api-access-nlcsj\") pod \"maas-controller-579cf7c6d-p7mck\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") " pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:31.942763 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:31.942725    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:32.309376 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:32.309298    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-p7mck"]
Nov 02 22:11:32.311963 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:11:32.311937    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e30cf89_5b1d_4c36_a049_4058f27451a5.slice/crio-f1894e13ff82b5d549dda12681fe4c442cfa58e9ae0f7a2cedee632380d011a5 WatchSource:0}: Error finding container f1894e13ff82b5d549dda12681fe4c442cfa58e9ae0f7a2cedee632380d011a5: Status 404 returned error can't find the container with id f1894e13ff82b5d549dda12681fe4c442cfa58e9ae0f7a2cedee632380d011a5
Nov 02 22:11:32.579622 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:32.579529    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-p7mck" event={"ID":"6e30cf89-5b1d-4c36-a049-4058f27451a5","Type":"ContainerStarted","Data":"f1894e13ff82b5d549dda12681fe4c442cfa58e9ae0f7a2cedee632380d011a5"}
Nov 02 22:11:35.596354 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:35.596316    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-p7mck" event={"ID":"6e30cf89-5b1d-4c36-a049-4058f27451a5","Type":"ContainerStarted","Data":"0565ebf007c85638fb28a83344706cf944800077156b743f9a97eac9a7a6fe14"}
Nov 02 22:11:35.596810 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:35.596521    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:35.711441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:35.711388    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-p7mck" podStartSLOduration=1.525774437 podStartE2EDuration="4.711359194s" totalImagesPullingTime="3.185584757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:11:35.710462677 +0000 UTC m=+367.994272385" watchObservedRunningTime="2026-11-02 22:11:35.711359194 +0000 UTC m=+367.995168880"
Nov 02 22:11:37.575155 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:11:37.575117    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:11:46.603694 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:46.603657    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:47.399421 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.399385    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-p7mck"]
Nov 02 22:11:47.399604 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.399572    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-p7mck" podUID="6e30cf89-5b1d-4c36-a049-4058f27451a5" containerName="manager" containerID="cri-o://0565ebf007c85638fb28a83344706cf944800077156b743f9a97eac9a7a6fe14" gracePeriod=30
Nov 02 22:11:47.653962 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.653902    5067 generic.go:356] "Generic (PLEG): container finished" podID="6e30cf89-5b1d-4c36-a049-4058f27451a5" containerID="0565ebf007c85638fb28a83344706cf944800077156b743f9a97eac9a7a6fe14" exitCode=0
Nov 02 22:11:47.654271 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.653971    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-p7mck" event={"ID":"6e30cf89-5b1d-4c36-a049-4058f27451a5","Type":"ContainerDied","Data":"0565ebf007c85638fb28a83344706cf944800077156b743f9a97eac9a7a6fe14"}
Nov 02 22:11:47.654271 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.654020    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-p7mck" event={"ID":"6e30cf89-5b1d-4c36-a049-4058f27451a5","Type":"ContainerDied","Data":"f1894e13ff82b5d549dda12681fe4c442cfa58e9ae0f7a2cedee632380d011a5"}
Nov 02 22:11:47.654271 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.654038    5067 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1894e13ff82b5d549dda12681fe4c442cfa58e9ae0f7a2cedee632380d011a5"
Nov 02 22:11:47.654589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.654570    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:47.770528 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.770496    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e30cf89-5b1d-4c36-a049-4058f27451a5-kube-api-access-nlcsj\" (UniqueName: \"kubernetes.io/projected/6e30cf89-5b1d-4c36-a049-4058f27451a5-kube-api-access-nlcsj\") pod \"6e30cf89-5b1d-4c36-a049-4058f27451a5\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") "
Nov 02 22:11:47.770712 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.770585    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-webhook-cert\") pod \"6e30cf89-5b1d-4c36-a049-4058f27451a5\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") "
Nov 02 22:11:47.770712 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.770631    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-metrics-certs\") pod \"6e30cf89-5b1d-4c36-a049-4058f27451a5\" (UID: \"6e30cf89-5b1d-4c36-a049-4058f27451a5\") "
Nov 02 22:11:47.773249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.773211    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e30cf89-5b1d-4c36-a049-4058f27451a5-kube-api-access-nlcsj" pod "6e30cf89-5b1d-4c36-a049-4058f27451a5" (UID: "6e30cf89-5b1d-4c36-a049-4058f27451a5"). InnerVolumeSpecName "kube-api-access-nlcsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:11:47.773249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.773238    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-webhook-cert" pod "6e30cf89-5b1d-4c36-a049-4058f27451a5" (UID: "6e30cf89-5b1d-4c36-a049-4058f27451a5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:11:47.773469 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.773402    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-metrics-certs" pod "6e30cf89-5b1d-4c36-a049-4058f27451a5" (UID: "6e30cf89-5b1d-4c36-a049-4058f27451a5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:11:47.871197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.871171    5067 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-metrics-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:11:47.871197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.871194    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlcsj\" (UniqueName: \"kubernetes.io/projected/6e30cf89-5b1d-4c36-a049-4058f27451a5-kube-api-access-nlcsj\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:11:47.871415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:47.871204    5067 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e30cf89-5b1d-4c36-a049-4058f27451a5-webhook-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:11:48.658926 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:48.658893    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-p7mck"
Nov 02 22:11:48.812929 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:48.812895    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-p7mck"]
Nov 02 22:11:48.859524 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:48.859485    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-p7mck"]
Nov 02 22:11:49.844208 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:11:49.844166    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e30cf89-5b1d-4c36-a049-4058f27451a5" path="/var/lib/kubelet/pods/6e30cf89-5b1d-4c36-a049-4058f27451a5/volumes"
Nov 02 22:12:06.921750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.921711    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"]
Nov 02 22:12:06.922239 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.922079    5067 cpu_manager.go:414] "containerMap: removing container" podUID="6e30cf89-5b1d-4c36-a049-4058f27451a5" containerName="manager"
Nov 02 22:12:06.922239 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.922090    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e30cf89-5b1d-4c36-a049-4058f27451a5" containerName="manager"
Nov 02 22:12:06.922239 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.922170    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e30cf89-5b1d-4c36-a049-4058f27451a5" containerName="manager"
Nov 02 22:12:06.929799 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.929768    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:06.979287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.979252    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 22:12:06.979758 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.979503    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 22:12:06.979758 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.979580    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 02 22:12:06.979758 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.979663    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-l4djg\""
Nov 02 22:12:06.979912 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.979839    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 22:12:06.981989 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:06.981967    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"]
Nov 02 22:12:07.059054 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.059021    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-openshift-service-ca\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.059255 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.059062    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-metrics-certs\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.059255 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.059206    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-maas-api-tls\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.059377 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.059286    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h58mj\" (UniqueName: \"kubernetes.io/projected/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-kube-api-access-h58mj\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.160607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.160565    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-maas-api-tls\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.160833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.160618    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h58mj\" (UniqueName: \"kubernetes.io/projected/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-kube-api-access-h58mj\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.160833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.160675    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-openshift-service-ca\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.160833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.160700    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-metrics-certs\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.161525 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.161499    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-openshift-service-ca\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.163326 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.163296    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-maas-api-tls\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.163444 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.163301    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-metrics-certs\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.222132 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.222095    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h58mj\" (UniqueName: \"kubernetes.io/projected/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-kube-api-access-h58mj\") pod \"maas-api-6d669c7fcd-k74jd\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.241204 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.241171    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:07.577841 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:12:07.577731    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:12:07.610556 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.610513    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"]
Nov 02 22:12:07.613624 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:12:07.613583    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f9cdfb6_0bc3_45e9_a35f_654be1c42a0e.slice/crio-a1a3ec7d466639a02ce34eb95dc3875f66159acf7bd78f671b6e4067633b161b WatchSource:0}: Error finding container a1a3ec7d466639a02ce34eb95dc3875f66159acf7bd78f671b6e4067633b161b: Status 404 returned error can't find the container with id a1a3ec7d466639a02ce34eb95dc3875f66159acf7bd78f671b6e4067633b161b
Nov 02 22:12:07.751034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:07.750989    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd" event={"ID":"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e","Type":"ContainerStarted","Data":"a1a3ec7d466639a02ce34eb95dc3875f66159acf7bd78f671b6e4067633b161b"}
Nov 02 22:12:10.767660 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:10.767625    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd" event={"ID":"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e","Type":"ContainerStarted","Data":"c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88"}
Nov 02 22:12:10.768103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:10.767745    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:10.882271 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:10.882199    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd" podStartSLOduration=2.687084673 podStartE2EDuration="4.882178532s" totalImagesPullingTime="2.195093859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:12:10.880020506 +0000 UTC m=+403.163830249" watchObservedRunningTime="2026-11-02 22:12:10.882178532 +0000 UTC m=+403.165988221"
Nov 02 22:12:15.288419 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.288389    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"]
Nov 02 22:12:15.291077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.291059    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:12:15.370865 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.370827    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5340b12-5a53-44c0-9804-9ee5e1cceae9-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-4qbjf\" (UID: \"c5340b12-5a53-44c0-9804-9ee5e1cceae9\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:12:15.371038 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.370928    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r6j2\" (UniqueName: \"kubernetes.io/projected/c5340b12-5a53-44c0-9804-9ee5e1cceae9-kube-api-access-9r6j2\") pod \"llmisvc-controller-manager-869d5dcc5d-4qbjf\" (UID: \"c5340b12-5a53-44c0-9804-9ee5e1cceae9\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:12:15.399220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.399189    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 02 22:12:15.399220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.399215    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 02 22:12:15.399447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.399234    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 02 22:12:15.399447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.399391    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-d6s28\""
Nov 02 22:12:15.400920 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.400900    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"]
Nov 02 22:12:15.471995 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.471962    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5340b12-5a53-44c0-9804-9ee5e1cceae9-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-4qbjf\" (UID: \"c5340b12-5a53-44c0-9804-9ee5e1cceae9\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:12:15.472182 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.472021    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r6j2\" (UniqueName: \"kubernetes.io/projected/c5340b12-5a53-44c0-9804-9ee5e1cceae9-kube-api-access-9r6j2\") pod \"llmisvc-controller-manager-869d5dcc5d-4qbjf\" (UID: \"c5340b12-5a53-44c0-9804-9ee5e1cceae9\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:12:15.474495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.474467    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5340b12-5a53-44c0-9804-9ee5e1cceae9-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-4qbjf\" (UID: \"c5340b12-5a53-44c0-9804-9ee5e1cceae9\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:12:15.529890 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.529852    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r6j2\" (UniqueName: \"kubernetes.io/projected/c5340b12-5a53-44c0-9804-9ee5e1cceae9-kube-api-access-9r6j2\") pod \"llmisvc-controller-manager-869d5dcc5d-4qbjf\" (UID: \"c5340b12-5a53-44c0-9804-9ee5e1cceae9\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:12:15.602023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.601937    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:12:15.988771 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:15.988728    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"]
Nov 02 22:12:15.992485 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:12:15.992455    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc5340b12_5a53_44c0_9804_9ee5e1cceae9.slice/crio-d8bf957058d1252dad952e9be1353e37975f203147d9efbb1dbcb3a445970915 WatchSource:0}: Error finding container d8bf957058d1252dad952e9be1353e37975f203147d9efbb1dbcb3a445970915: Status 404 returned error can't find the container with id d8bf957058d1252dad952e9be1353e37975f203147d9efbb1dbcb3a445970915
Nov 02 22:12:16.778094 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:16.778010    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:16.797176 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:16.797144    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf" event={"ID":"c5340b12-5a53-44c0-9804-9ee5e1cceae9","Type":"ContainerStarted","Data":"d8bf957058d1252dad952e9be1353e37975f203147d9efbb1dbcb3a445970915"}
Nov 02 22:12:19.813030 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:19.812939    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf" event={"ID":"c5340b12-5a53-44c0-9804-9ee5e1cceae9","Type":"ContainerStarted","Data":"112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30"}
Nov 02 22:12:19.813541 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:19.813036    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:12:19.814600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:19.814577    5067 generic.go:356] "Generic (PLEG): container finished" podID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerID="f3432dddba01ac5a6cabae65d3db995397765409a6110439e0c85c9336e0ed9e" exitCode=0
Nov 02 22:12:19.814734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:19.814653    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" event={"ID":"0ddee653-e8f0-42a6-9c14-3361ccc6dc97","Type":"ContainerDied","Data":"f3432dddba01ac5a6cabae65d3db995397765409a6110439e0c85c9336e0ed9e"}
Nov 02 22:12:19.815031 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:19.815012    5067 scope.go:122] "RemoveContainer" containerID="f3432dddba01ac5a6cabae65d3db995397765409a6110439e0c85c9336e0ed9e"
Nov 02 22:12:19.816114 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:19.816086    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:12:19.816190 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:19.816156    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:12:19.816227 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:19.816188    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:12:19.923008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:19.922953    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf" podStartSLOduration=1.7244619 podStartE2EDuration="4.922937617s" totalImagesPullingTime="3.198475717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:12:19.921644678 +0000 UTC m=+412.205454388" watchObservedRunningTime="2026-11-02 22:12:19.922937617 +0000 UTC m=+412.206747304"
Nov 02 22:12:20.822656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:20.822619    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" event={"ID":"0ddee653-e8f0-42a6-9c14-3361ccc6dc97","Type":"ContainerStarted","Data":"763cd89501003e72ec6c2a04e41db46a182beec439ca10181a71111be1c0c93c"}
Nov 02 22:12:21.185388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:21.185278    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:12:21.185542 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:21.185494    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-7sfm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Nov 02 22:12:21.185586 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:21.185554    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Nov 02 22:12:22.185379 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:22.185333    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-7sfm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Nov 02 22:12:22.185782 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:22.185425    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Nov 02 22:12:23.185514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:23.185474    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-7sfm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Nov 02 22:12:23.185909 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:23.185535    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Nov 02 22:12:24.185692 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:24.185657    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-7sfm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Nov 02 22:12:24.186078 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:24.185709    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Nov 02 22:12:25.185535 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:25.185497    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-7sfm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Nov 02 22:12:25.185727 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:25.185553    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Nov 02 22:12:26.189890 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:26.189802    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:12:26.190244 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:26.190094    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:12:26.190923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:26.190903    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:12:28.024123 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.024092    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cc8f7bd77-vm7jq"]
Nov 02 22:12:28.027762 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.027732    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cc8f7bd77-vm7jq"
Nov 02 22:12:28.081491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.081455    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cc8f7bd77-vm7jq"]
Nov 02 22:12:28.106310 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.106285    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsns2\" (UniqueName: \"kubernetes.io/projected/e4eec667-c8de-4a43-8c74-7f462818451a-kube-api-access-nsns2\") pod \"authorino-cc8f7bd77-vm7jq\" (UID: \"e4eec667-c8de-4a43-8c74-7f462818451a\") " pod="kuadrant-system/authorino-cc8f7bd77-vm7jq"
Nov 02 22:12:28.207620 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.207578    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsns2\" (UniqueName: \"kubernetes.io/projected/e4eec667-c8de-4a43-8c74-7f462818451a-kube-api-access-nsns2\") pod \"authorino-cc8f7bd77-vm7jq\" (UID: \"e4eec667-c8de-4a43-8c74-7f462818451a\") " pod="kuadrant-system/authorino-cc8f7bd77-vm7jq"
Nov 02 22:12:28.265806 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.265771    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsns2\" (UniqueName: \"kubernetes.io/projected/e4eec667-c8de-4a43-8c74-7f462818451a-kube-api-access-nsns2\") pod \"authorino-cc8f7bd77-vm7jq\" (UID: \"e4eec667-c8de-4a43-8c74-7f462818451a\") " pod="kuadrant-system/authorino-cc8f7bd77-vm7jq"
Nov 02 22:12:28.340733 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.340655    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cc8f7bd77-vm7jq"
Nov 02 22:12:28.443062 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.443024    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cc8f7bd77-vm7jq"]
Nov 02 22:12:28.475170 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.475137    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8446fd7595-skkz2"]
Nov 02 22:12:28.479974 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.479956    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8446fd7595-skkz2"
Nov 02 22:12:28.511435 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.511400    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-tls-cert\") pod \"authorino-8446fd7595-skkz2\" (UID: \"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a\") " pod="kuadrant-system/authorino-8446fd7595-skkz2"
Nov 02 22:12:28.511435 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.511436    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b8v9\" (UniqueName: \"kubernetes.io/projected/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-kube-api-access-4b8v9\") pod \"authorino-8446fd7595-skkz2\" (UID: \"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a\") " pod="kuadrant-system/authorino-8446fd7595-skkz2"
Nov 02 22:12:28.530739 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.530704    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 02 22:12:28.532966 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.532942    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8446fd7595-skkz2"]
Nov 02 22:12:28.612500 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.612420    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-tls-cert\") pod \"authorino-8446fd7595-skkz2\" (UID: \"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a\") " pod="kuadrant-system/authorino-8446fd7595-skkz2"
Nov 02 22:12:28.612500 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.612456    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b8v9\" (UniqueName: \"kubernetes.io/projected/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-kube-api-access-4b8v9\") pod \"authorino-8446fd7595-skkz2\" (UID: \"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a\") " pod="kuadrant-system/authorino-8446fd7595-skkz2"
Nov 02 22:12:28.614943 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.614916    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-tls-cert\") pod \"authorino-8446fd7595-skkz2\" (UID: \"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a\") " pod="kuadrant-system/authorino-8446fd7595-skkz2"
Nov 02 22:12:28.671279 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.671241    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b8v9\" (UniqueName: \"kubernetes.io/projected/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-kube-api-access-4b8v9\") pod \"authorino-8446fd7595-skkz2\" (UID: \"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a\") " pod="kuadrant-system/authorino-8446fd7595-skkz2"
Nov 02 22:12:28.704676 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.704576    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cc8f7bd77-vm7jq"]
Nov 02 22:12:28.707415 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:12:28.707385    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4eec667_c8de_4a43_8c74_7f462818451a.slice/crio-5986864a9d97e60263ebb5d95c56f751c95f48efbde10d24f06b35b0ebd278a4 WatchSource:0}: Error finding container 5986864a9d97e60263ebb5d95c56f751c95f48efbde10d24f06b35b0ebd278a4: Status 404 returned error can't find the container with id 5986864a9d97e60263ebb5d95c56f751c95f48efbde10d24f06b35b0ebd278a4
Nov 02 22:12:28.791184 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.791153    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8446fd7595-skkz2"
Nov 02 22:12:28.886974 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:28.886897    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cc8f7bd77-vm7jq" event={"ID":"e4eec667-c8de-4a43-8c74-7f462818451a","Type":"ContainerStarted","Data":"5986864a9d97e60263ebb5d95c56f751c95f48efbde10d24f06b35b0ebd278a4"}
Nov 02 22:12:29.154481 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:29.154407    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8446fd7595-skkz2"]
Nov 02 22:12:29.157068 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:12:29.157042    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76b889a5_ed3c_4ef6_96d0_66b9c9d69e4a.slice/crio-8c9494828d87b31b2d654e81222639dee33f760c5739b65faeb8905c29f9133f WatchSource:0}: Error finding container 8c9494828d87b31b2d654e81222639dee33f760c5739b65faeb8905c29f9133f: Status 404 returned error can't find the container with id 8c9494828d87b31b2d654e81222639dee33f760c5739b65faeb8905c29f9133f
Nov 02 22:12:29.892810 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:29.892774    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8446fd7595-skkz2" event={"ID":"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a","Type":"ContainerStarted","Data":"8c9494828d87b31b2d654e81222639dee33f760c5739b65faeb8905c29f9133f"}
Nov 02 22:12:30.902834 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:30.902730    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8446fd7595-skkz2" event={"ID":"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a","Type":"ContainerStarted","Data":"52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401"}
Nov 02 22:12:30.905344 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:30.905289    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cc8f7bd77-vm7jq" event={"ID":"e4eec667-c8de-4a43-8c74-7f462818451a","Type":"ContainerStarted","Data":"59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885"}
Nov 02 22:12:30.905344 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:30.905325    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-cc8f7bd77-vm7jq" podUID="e4eec667-c8de-4a43-8c74-7f462818451a" containerName="authorino" containerID="cri-o://59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885" gracePeriod=30
Nov 02 22:12:31.012917 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.012864    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-8446fd7595-skkz2" podStartSLOduration=1.763697154 podStartE2EDuration="3.012849222s" totalImagesPullingTime="1.249152068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:12:31.011451731 +0000 UTC m=+423.295261439" watchObservedRunningTime="2026-11-02 22:12:31.012849222 +0000 UTC m=+423.296658907"
Nov 02 22:12:31.040721 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.040680    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2d9jk"]
Nov 02 22:12:31.040943 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.040919    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk" podUID="3766427f-9f22-4e73-afa7-5a0b7faa237b" containerName="authorino" containerID="cri-o://0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a" gracePeriod=30
Nov 02 22:12:31.122323 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.122258    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-cc8f7bd77-vm7jq" podStartSLOduration=2.605049949 podStartE2EDuration="4.12223665s" totalImagesPullingTime="1.517186701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:12:31.11986933 +0000 UTC m=+423.403679034" watchObservedRunningTime="2026-11-02 22:12:31.12223665 +0000 UTC m=+423.406046341"
Nov 02 22:12:31.163750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.163725    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cc8f7bd77-vm7jq"
Nov 02 22:12:31.240719 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.237711    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4eec667-c8de-4a43-8c74-7f462818451a-kube-api-access-nsns2\" (UniqueName: \"kubernetes.io/projected/e4eec667-c8de-4a43-8c74-7f462818451a-kube-api-access-nsns2\") pod \"e4eec667-c8de-4a43-8c74-7f462818451a\" (UID: \"e4eec667-c8de-4a43-8c74-7f462818451a\") "
Nov 02 22:12:31.241503 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.241472    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4eec667-c8de-4a43-8c74-7f462818451a-kube-api-access-nsns2" pod "e4eec667-c8de-4a43-8c74-7f462818451a" (UID: "e4eec667-c8de-4a43-8c74-7f462818451a"). InnerVolumeSpecName "kube-api-access-nsns2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:31.287070 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.287043    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk"
Nov 02 22:12:31.339179 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.339140    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3766427f-9f22-4e73-afa7-5a0b7faa237b-kube-api-access-xw64x\" (UniqueName: \"kubernetes.io/projected/3766427f-9f22-4e73-afa7-5a0b7faa237b-kube-api-access-xw64x\") pod \"3766427f-9f22-4e73-afa7-5a0b7faa237b\" (UID: \"3766427f-9f22-4e73-afa7-5a0b7faa237b\") "
Nov 02 22:12:31.339418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.339395    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsns2\" (UniqueName: \"kubernetes.io/projected/e4eec667-c8de-4a43-8c74-7f462818451a-kube-api-access-nsns2\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:12:31.342004 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.341976    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3766427f-9f22-4e73-afa7-5a0b7faa237b-kube-api-access-xw64x" pod "3766427f-9f22-4e73-afa7-5a0b7faa237b" (UID: "3766427f-9f22-4e73-afa7-5a0b7faa237b"). InnerVolumeSpecName "kube-api-access-xw64x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:31.440865 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.440827    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw64x\" (UniqueName: \"kubernetes.io/projected/3766427f-9f22-4e73-afa7-5a0b7faa237b-kube-api-access-xw64x\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:12:31.911315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.911272    5067 generic.go:356] "Generic (PLEG): container finished" podID="e4eec667-c8de-4a43-8c74-7f462818451a" containerID="59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885" exitCode=0
Nov 02 22:12:31.911315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.911318    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cc8f7bd77-vm7jq"
Nov 02 22:12:31.911833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.911359    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cc8f7bd77-vm7jq" event={"ID":"e4eec667-c8de-4a43-8c74-7f462818451a","Type":"ContainerDied","Data":"59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885"}
Nov 02 22:12:31.911833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.911431    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cc8f7bd77-vm7jq" event={"ID":"e4eec667-c8de-4a43-8c74-7f462818451a","Type":"ContainerDied","Data":"5986864a9d97e60263ebb5d95c56f751c95f48efbde10d24f06b35b0ebd278a4"}
Nov 02 22:12:31.911833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.911455    5067 scope.go:122] "RemoveContainer" containerID="59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885"
Nov 02 22:12:31.912865 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.912844    5067 generic.go:356] "Generic (PLEG): container finished" podID="3766427f-9f22-4e73-afa7-5a0b7faa237b" containerID="0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a" exitCode=0
Nov 02 22:12:31.912962 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.912890    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk"
Nov 02 22:12:31.912962 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.912929    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk" event={"ID":"3766427f-9f22-4e73-afa7-5a0b7faa237b","Type":"ContainerDied","Data":"0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a"}
Nov 02 22:12:31.913072 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.912966    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-2d9jk" event={"ID":"3766427f-9f22-4e73-afa7-5a0b7faa237b","Type":"ContainerDied","Data":"2ac4d0ab56acae9e785ab4bb1bc772ad5f1c3df73c5d1c2da1c068b2fe070bbd"}
Nov 02 22:12:31.920148 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.920130    5067 scope.go:122] "RemoveContainer" containerID="59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885"
Nov 02 22:12:31.920427 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:12:31.920406    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885\": container with ID starting with 59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885 not found: ID does not exist" containerID="59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885"
Nov 02 22:12:31.920475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.920435    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885"} err="failed to get container status \"59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885\": rpc error: code = NotFound desc = could not find container \"59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885\": container with ID starting with 59d7090ac467bc31085df4edadfb92c4b4d378051cac5b4266503b713aa19885 not found: ID does not exist"
Nov 02 22:12:31.920475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.920452    5067 scope.go:122] "RemoveContainer" containerID="0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a"
Nov 02 22:12:31.928314 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.928298    5067 scope.go:122] "RemoveContainer" containerID="0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a"
Nov 02 22:12:31.928600 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:12:31.928581    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a\": container with ID starting with 0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a not found: ID does not exist" containerID="0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a"
Nov 02 22:12:31.928643 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:31.928606    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a"} err="failed to get container status \"0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a\": rpc error: code = NotFound desc = could not find container \"0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a\": container with ID starting with 0cd393b4b67ab72b24286d5c5eab969d7e05bb602674eff086719aa9e0c3ba6a not found: ID does not exist"
Nov 02 22:12:32.064617 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:32.064580    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cc8f7bd77-vm7jq"]
Nov 02 22:12:32.071164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:32.071138    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cc8f7bd77-vm7jq"]
Nov 02 22:12:32.173152 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:32.173057    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2d9jk"]
Nov 02 22:12:32.177322 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:32.177297    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-2d9jk"]
Nov 02 22:12:33.844163 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:33.844129    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3766427f-9f22-4e73-afa7-5a0b7faa237b" path="/var/lib/kubelet/pods/3766427f-9f22-4e73-afa7-5a0b7faa237b/volumes"
Nov 02 22:12:33.844569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:33.844491    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4eec667-c8de-4a43-8c74-7f462818451a" path="/var/lib/kubelet/pods/e4eec667-c8de-4a43-8c74-7f462818451a/volumes"
Nov 02 22:12:37.577053 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:12:37.577019    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:12:38.650816 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:38.650778    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"]
Nov 02 22:12:38.651393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:38.651047    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd" podUID="2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e" containerName="maas-api" containerID="cri-o://c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88" gracePeriod=30
Nov 02 22:12:38.917701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:38.917677    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:38.949415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:38.949383    5067 generic.go:356] "Generic (PLEG): container finished" podID="2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e" containerID="c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88" exitCode=0
Nov 02 22:12:38.949531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:38.949430    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"
Nov 02 22:12:38.949531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:38.949451    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd" event={"ID":"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e","Type":"ContainerDied","Data":"c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88"}
Nov 02 22:12:38.949531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:38.949506    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd" event={"ID":"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e","Type":"ContainerDied","Data":"a1a3ec7d466639a02ce34eb95dc3875f66159acf7bd78f671b6e4067633b161b"}
Nov 02 22:12:38.949531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:38.949518    5067 scope.go:122] "RemoveContainer" containerID="c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88"
Nov 02 22:12:38.957967 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:38.957950    5067 scope.go:122] "RemoveContainer" containerID="c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88"
Nov 02 22:12:38.958242 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:12:38.958225    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88\": container with ID starting with c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88 not found: ID does not exist" containerID="c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88"
Nov 02 22:12:38.958280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:38.958257    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88"} err="failed to get container status \"c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88\": rpc error: code = NotFound desc = could not find container \"c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88\": container with ID starting with c65bdffd25613ec02be7487b1ef22faa7ce7dc3c34a18e3047412391a8c44b88 not found: ID does not exist"
Nov 02 22:12:39.011171 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.011137    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-metrics-certs\") pod \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") "
Nov 02 22:12:39.011171 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.011172    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-maas-api-tls\") pod \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") "
Nov 02 22:12:39.011358 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.011307    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-kube-api-access-h58mj\" (UniqueName: \"kubernetes.io/projected/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-kube-api-access-h58mj\") pod \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") "
Nov 02 22:12:39.011431 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.011400    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-openshift-service-ca\") pod \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\" (UID: \"2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e\") "
Nov 02 22:12:39.011797 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.011772    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-openshift-service-ca" pod "2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e" (UID: "2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:12:39.013589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.013563    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-kube-api-access-h58mj" pod "2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e" (UID: "2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e"). InnerVolumeSpecName "kube-api-access-h58mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:12:39.013743 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.013726    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-maas-api-tls" pod "2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e" (UID: "2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:12:39.013845 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.013826    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-metrics-certs" pod "2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e" (UID: "2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:12:39.112414 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.112354    5067 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-metrics-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:12:39.112414 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.112408    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-maas-api-tls\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:12:39.112414 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.112419    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h58mj\" (UniqueName: \"kubernetes.io/projected/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-kube-api-access-h58mj\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:12:39.112656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.112430    5067 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e-openshift-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:12:39.364503 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.364465    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"]
Nov 02 22:12:39.370971 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.370941    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-k74jd"]
Nov 02 22:12:39.844022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:39.843988    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e" path="/var/lib/kubelet/pods/2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e/volumes"
Nov 02 22:12:50.825585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:12:50.825550    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:13:07.577533 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:13:07.577491    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:13:32.510737 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.510701    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"]
Nov 02 22:13:32.511220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.511086    5067 cpu_manager.go:414] "containerMap: removing container" podUID="3766427f-9f22-4e73-afa7-5a0b7faa237b" containerName="authorino"
Nov 02 22:13:32.511220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.511097    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3766427f-9f22-4e73-afa7-5a0b7faa237b" containerName="authorino"
Nov 02 22:13:32.511220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.511105    5067 cpu_manager.go:414] "containerMap: removing container" podUID="e4eec667-c8de-4a43-8c74-7f462818451a" containerName="authorino"
Nov 02 22:13:32.511220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.511110    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4eec667-c8de-4a43-8c74-7f462818451a" containerName="authorino"
Nov 02 22:13:32.511220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.511121    5067 cpu_manager.go:414] "containerMap: removing container" podUID="2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e" containerName="maas-api"
Nov 02 22:13:32.511220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.511127    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e" containerName="maas-api"
Nov 02 22:13:32.511220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.511193    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="3766427f-9f22-4e73-afa7-5a0b7faa237b" containerName="authorino"
Nov 02 22:13:32.511220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.511200    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4eec667-c8de-4a43-8c74-7f462818451a" containerName="authorino"
Nov 02 22:13:32.511220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.511208    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f9cdfb6-0bc3-45e9-a35f-654be1c42a0e" containerName="maas-api"
Nov 02 22:13:32.515776 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.515758    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"
Nov 02 22:13:32.570132 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.570085    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"]
Nov 02 22:13:32.587037 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.587004    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f703922-4ab8-4d37-b8dc-03cbbf0eea31-cert\") pod \"llmisvc-controller-manager-6466c554d8-pjd76\" (UID: \"6f703922-4ab8-4d37-b8dc-03cbbf0eea31\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"
Nov 02 22:13:32.587197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.587133    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmwsb\" (UniqueName: \"kubernetes.io/projected/6f703922-4ab8-4d37-b8dc-03cbbf0eea31-kube-api-access-kmwsb\") pod \"llmisvc-controller-manager-6466c554d8-pjd76\" (UID: \"6f703922-4ab8-4d37-b8dc-03cbbf0eea31\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"
Nov 02 22:13:32.687968 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.687922    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmwsb\" (UniqueName: \"kubernetes.io/projected/6f703922-4ab8-4d37-b8dc-03cbbf0eea31-kube-api-access-kmwsb\") pod \"llmisvc-controller-manager-6466c554d8-pjd76\" (UID: \"6f703922-4ab8-4d37-b8dc-03cbbf0eea31\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"
Nov 02 22:13:32.688152 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.688003    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f703922-4ab8-4d37-b8dc-03cbbf0eea31-cert\") pod \"llmisvc-controller-manager-6466c554d8-pjd76\" (UID: \"6f703922-4ab8-4d37-b8dc-03cbbf0eea31\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"
Nov 02 22:13:32.690555 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.690528    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f703922-4ab8-4d37-b8dc-03cbbf0eea31-cert\") pod \"llmisvc-controller-manager-6466c554d8-pjd76\" (UID: \"6f703922-4ab8-4d37-b8dc-03cbbf0eea31\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"
Nov 02 22:13:32.746191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.746152    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmwsb\" (UniqueName: \"kubernetes.io/projected/6f703922-4ab8-4d37-b8dc-03cbbf0eea31-kube-api-access-kmwsb\") pod \"llmisvc-controller-manager-6466c554d8-pjd76\" (UID: \"6f703922-4ab8-4d37-b8dc-03cbbf0eea31\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"
Nov 02 22:13:32.828682 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:32.828591    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"
Nov 02 22:13:33.195183 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:33.195003    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"]
Nov 02 22:13:33.198039 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:13:33.198011    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f703922_4ab8_4d37_b8dc_03cbbf0eea31.slice/crio-84ea2377264b393c62ae9b6451f397fa36d484c1525b49498fee9930e5266019 WatchSource:0}: Error finding container 84ea2377264b393c62ae9b6451f397fa36d484c1525b49498fee9930e5266019: Status 404 returned error can't find the container with id 84ea2377264b393c62ae9b6451f397fa36d484c1525b49498fee9930e5266019
Nov 02 22:13:33.205596 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:33.205561    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76" event={"ID":"6f703922-4ab8-4d37-b8dc-03cbbf0eea31","Type":"ContainerStarted","Data":"84ea2377264b393c62ae9b6451f397fa36d484c1525b49498fee9930e5266019"}
Nov 02 22:13:34.211824 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:34.211789    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76" event={"ID":"6f703922-4ab8-4d37-b8dc-03cbbf0eea31","Type":"ContainerStarted","Data":"9cee4f9e82ccad25b6212a6279300fbc2a832101acc18bcbef81fed74a464a84"}
Nov 02 22:13:34.212194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:34.211897    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"
Nov 02 22:13:34.321639 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:13:34.321581    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76" podStartSLOduration=1.402290102 podStartE2EDuration="2.321565154s" totalImagesPullingTime="919.275052ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:13:34.321089953 +0000 UTC m=+486.604899666" watchObservedRunningTime="2026-11-02 22:13:34.321565154 +0000 UTC m=+486.605374841"
Nov 02 22:13:37.575673 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:13:37.575632    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:14:05.219683 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:05.219649    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-pjd76"
Nov 02 22:14:05.377946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:05.377913    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"]
Nov 02 22:14:05.378204 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:05.378149    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf" podUID="c5340b12-5a53-44c0-9804-9ee5e1cceae9" containerName="manager" containerID="cri-o://112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30" gracePeriod=30
Nov 02 22:14:05.623686 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:05.623659    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:14:05.677180 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:05.677140    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5340b12-5a53-44c0-9804-9ee5e1cceae9-kube-api-access-9r6j2\" (UniqueName: \"kubernetes.io/projected/c5340b12-5a53-44c0-9804-9ee5e1cceae9-kube-api-access-9r6j2\") pod \"c5340b12-5a53-44c0-9804-9ee5e1cceae9\" (UID: \"c5340b12-5a53-44c0-9804-9ee5e1cceae9\") "
Nov 02 22:14:05.677402 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:05.677236    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c5340b12-5a53-44c0-9804-9ee5e1cceae9-cert\" (UniqueName: \"kubernetes.io/secret/c5340b12-5a53-44c0-9804-9ee5e1cceae9-cert\") pod \"c5340b12-5a53-44c0-9804-9ee5e1cceae9\" (UID: \"c5340b12-5a53-44c0-9804-9ee5e1cceae9\") "
Nov 02 22:14:05.679273 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:05.679239    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5340b12-5a53-44c0-9804-9ee5e1cceae9-cert" pod "c5340b12-5a53-44c0-9804-9ee5e1cceae9" (UID: "c5340b12-5a53-44c0-9804-9ee5e1cceae9"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:14:05.679402 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:05.679292    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5340b12-5a53-44c0-9804-9ee5e1cceae9-kube-api-access-9r6j2" pod "c5340b12-5a53-44c0-9804-9ee5e1cceae9" (UID: "c5340b12-5a53-44c0-9804-9ee5e1cceae9"). InnerVolumeSpecName "kube-api-access-9r6j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:14:05.778330 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:05.778258    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r6j2\" (UniqueName: \"kubernetes.io/projected/c5340b12-5a53-44c0-9804-9ee5e1cceae9-kube-api-access-9r6j2\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:14:05.778330 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:05.778284    5067 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5340b12-5a53-44c0-9804-9ee5e1cceae9-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:14:06.373967 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:06.373929    5067 generic.go:356] "Generic (PLEG): container finished" podID="c5340b12-5a53-44c0-9804-9ee5e1cceae9" containerID="112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30" exitCode=0
Nov 02 22:14:06.374448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:06.373978    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"
Nov 02 22:14:06.374448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:06.374013    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf" event={"ID":"c5340b12-5a53-44c0-9804-9ee5e1cceae9","Type":"ContainerDied","Data":"112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30"}
Nov 02 22:14:06.374448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:06.374081    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf" event={"ID":"c5340b12-5a53-44c0-9804-9ee5e1cceae9","Type":"ContainerDied","Data":"d8bf957058d1252dad952e9be1353e37975f203147d9efbb1dbcb3a445970915"}
Nov 02 22:14:06.374448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:06.374102    5067 scope.go:122] "RemoveContainer" containerID="112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30"
Nov 02 22:14:06.382678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:06.382492    5067 scope.go:122] "RemoveContainer" containerID="112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30"
Nov 02 22:14:06.382792 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:14:06.382771    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30\": container with ID starting with 112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30 not found: ID does not exist" containerID="112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30"
Nov 02 22:14:06.382835 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:06.382801    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30"} err="failed to get container status \"112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30\": rpc error: code = NotFound desc = could not find container \"112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30\": container with ID starting with 112de3dfa58643998d25d7ffb50afd763f8fc3ac1418419add3ea48f2146be30 not found: ID does not exist"
Nov 02 22:14:06.532392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:06.532347    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"]
Nov 02 22:14:06.578710 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:06.578680    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf"]
Nov 02 22:14:07.576433 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:14:07.576393    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:14:07.844585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:14:07.844498    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5340b12-5a53-44c0-9804-9ee5e1cceae9" path="/var/lib/kubelet/pods/c5340b12-5a53-44c0-9804-9ee5e1cceae9/volumes"
Nov 02 22:14:37.575070 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:14:37.575032    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:15:07.576290 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:15:07.576245    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:15:19.395400 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.395350    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7dc8f7b988-ccqln"]
Nov 02 22:15:19.395858 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.395733    5067 cpu_manager.go:414] "containerMap: removing container" podUID="c5340b12-5a53-44c0-9804-9ee5e1cceae9" containerName="manager"
Nov 02 22:15:19.395858 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.395745    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5340b12-5a53-44c0-9804-9ee5e1cceae9" containerName="manager"
Nov 02 22:15:19.395858 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.395825    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5340b12-5a53-44c0-9804-9ee5e1cceae9" containerName="manager"
Nov 02 22:15:19.398854 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.398832    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dc8f7b988-ccqln"
Nov 02 22:15:19.453835 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.453798    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dc8f7b988-ccqln"]
Nov 02 22:15:19.482630 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.482595    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/36d566dd-6bc5-428c-ac92-ed9944c78c72-tls-cert\") pod \"authorino-7dc8f7b988-ccqln\" (UID: \"36d566dd-6bc5-428c-ac92-ed9944c78c72\") " pod="kuadrant-system/authorino-7dc8f7b988-ccqln"
Nov 02 22:15:19.482796 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.482722    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzbd9\" (UniqueName: \"kubernetes.io/projected/36d566dd-6bc5-428c-ac92-ed9944c78c72-kube-api-access-tzbd9\") pod \"authorino-7dc8f7b988-ccqln\" (UID: \"36d566dd-6bc5-428c-ac92-ed9944c78c72\") " pod="kuadrant-system/authorino-7dc8f7b988-ccqln"
Nov 02 22:15:19.583671 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.583630    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzbd9\" (UniqueName: \"kubernetes.io/projected/36d566dd-6bc5-428c-ac92-ed9944c78c72-kube-api-access-tzbd9\") pod \"authorino-7dc8f7b988-ccqln\" (UID: \"36d566dd-6bc5-428c-ac92-ed9944c78c72\") " pod="kuadrant-system/authorino-7dc8f7b988-ccqln"
Nov 02 22:15:19.583861 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.583729    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/36d566dd-6bc5-428c-ac92-ed9944c78c72-tls-cert\") pod \"authorino-7dc8f7b988-ccqln\" (UID: \"36d566dd-6bc5-428c-ac92-ed9944c78c72\") " pod="kuadrant-system/authorino-7dc8f7b988-ccqln"
Nov 02 22:15:19.586130 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.586102    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/36d566dd-6bc5-428c-ac92-ed9944c78c72-tls-cert\") pod \"authorino-7dc8f7b988-ccqln\" (UID: \"36d566dd-6bc5-428c-ac92-ed9944c78c72\") " pod="kuadrant-system/authorino-7dc8f7b988-ccqln"
Nov 02 22:15:19.642098 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.642061    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzbd9\" (UniqueName: \"kubernetes.io/projected/36d566dd-6bc5-428c-ac92-ed9944c78c72-kube-api-access-tzbd9\") pod \"authorino-7dc8f7b988-ccqln\" (UID: \"36d566dd-6bc5-428c-ac92-ed9944c78c72\") " pod="kuadrant-system/authorino-7dc8f7b988-ccqln"
Nov 02 22:15:19.710343 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:19.710267    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dc8f7b988-ccqln"
Nov 02 22:15:20.077923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:20.077889    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dc8f7b988-ccqln"]
Nov 02 22:15:20.080860 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:15:20.080835    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36d566dd_6bc5_428c_ac92_ed9944c78c72.slice/crio-a949b03e7ffa1957b71e3fbd29aa1c012d3a3d52e7aa2eb88e55c0de60997c5a WatchSource:0}: Error finding container a949b03e7ffa1957b71e3fbd29aa1c012d3a3d52e7aa2eb88e55c0de60997c5a: Status 404 returned error can't find the container with id a949b03e7ffa1957b71e3fbd29aa1c012d3a3d52e7aa2eb88e55c0de60997c5a
Nov 02 22:15:20.742751 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:20.742712    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dc8f7b988-ccqln" event={"ID":"36d566dd-6bc5-428c-ac92-ed9944c78c72","Type":"ContainerStarted","Data":"a949b03e7ffa1957b71e3fbd29aa1c012d3a3d52e7aa2eb88e55c0de60997c5a"}
Nov 02 22:15:22.756175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:22.756128    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dc8f7b988-ccqln" event={"ID":"36d566dd-6bc5-428c-ac92-ed9944c78c72","Type":"ContainerStarted","Data":"1ee0c72ec6c48d806339a6faab5dce738b9e4b9f52929397d8da0b73605010b1"}
Nov 02 22:15:22.864554 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:22.864502    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-7dc8f7b988-ccqln" podStartSLOduration=2.146373801 podStartE2EDuration="3.864488958s" totalImagesPullingTime="1.718115157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:15:22.864173137 +0000 UTC m=+595.147982825" watchObservedRunningTime="2026-11-02 22:15:22.864488958 +0000 UTC m=+595.148298644"
Nov 02 22:15:22.915061 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:22.915016    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8446fd7595-skkz2"]
Nov 02 22:15:22.915555 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:22.915523    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-8446fd7595-skkz2" podUID="76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a" containerName="authorino" containerID="cri-o://52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401" gracePeriod=30
Nov 02 22:15:23.184332 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.184301    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8446fd7595-skkz2"
Nov 02 22:15:23.321686 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.321587    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-tls-cert\" (UniqueName: \"kubernetes.io/secret/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-tls-cert\") pod \"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a\" (UID: \"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a\") "
Nov 02 22:15:23.321686 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.321665    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-kube-api-access-4b8v9\" (UniqueName: \"kubernetes.io/projected/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-kube-api-access-4b8v9\") pod \"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a\" (UID: \"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a\") "
Nov 02 22:15:23.324053 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.324019    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-kube-api-access-4b8v9" pod "76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a" (UID: "76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a"). InnerVolumeSpecName "kube-api-access-4b8v9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:15:23.334486 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.334448    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-tls-cert" pod "76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a" (UID: "76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:15:23.423162 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.423119    5067 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-tls-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:15:23.423162 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.423163    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b8v9\" (UniqueName: \"kubernetes.io/projected/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a-kube-api-access-4b8v9\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:15:23.762778 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.762738    5067 generic.go:356] "Generic (PLEG): container finished" podID="76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a" containerID="52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401" exitCode=0
Nov 02 22:15:23.763212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.762785    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8446fd7595-skkz2"
Nov 02 22:15:23.763212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.762833    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8446fd7595-skkz2" event={"ID":"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a","Type":"ContainerDied","Data":"52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401"}
Nov 02 22:15:23.763212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.762896    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8446fd7595-skkz2" event={"ID":"76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a","Type":"ContainerDied","Data":"8c9494828d87b31b2d654e81222639dee33f760c5739b65faeb8905c29f9133f"}
Nov 02 22:15:23.763212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.762925    5067 scope.go:122] "RemoveContainer" containerID="52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401"
Nov 02 22:15:23.772685 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.772667    5067 scope.go:122] "RemoveContainer" containerID="52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401"
Nov 02 22:15:23.772972 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:15:23.772953    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401\": container with ID starting with 52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401 not found: ID does not exist" containerID="52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401"
Nov 02 22:15:23.773031 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.772981    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401"} err="failed to get container status \"52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401\": rpc error: code = NotFound desc = could not find container \"52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401\": container with ID starting with 52e491396758891360bd2f0ff3af9e4bf48367adca68218bf4bf57c5bebce401 not found: ID does not exist"
Nov 02 22:15:23.916662 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.916626    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8446fd7595-skkz2"]
Nov 02 22:15:23.921626 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:23.921597    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8446fd7595-skkz2"]
Nov 02 22:15:24.742416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.742376    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"]
Nov 02 22:15:24.742777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.742761    5067 cpu_manager.go:414] "containerMap: removing container" podUID="76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a" containerName="authorino"
Nov 02 22:15:24.742777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.742776    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a" containerName="authorino"
Nov 02 22:15:24.742927 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.742873    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a" containerName="authorino"
Nov 02 22:15:24.747001 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.746976    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.836843 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.836797    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.837301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.836854    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.837301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.836913    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.837301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.837030    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.837301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.837085    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h4s6\" (UniqueName: \"kubernetes.io/projected/790b1bdf-449a-4671-b640-97990dd0d405-kube-api-access-9h4s6\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.837301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.837172    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/790b1bdf-449a-4671-b640-97990dd0d405-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.837301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.837275    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/790b1bdf-449a-4671-b640-97990dd0d405-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.837578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.837315    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/790b1bdf-449a-4671-b640-97990dd0d405-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.837578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.837402    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.938314 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.938277    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.938542 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.938331    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h4s6\" (UniqueName: \"kubernetes.io/projected/790b1bdf-449a-4671-b640-97990dd0d405-kube-api-access-9h4s6\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.938542 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.938360    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/790b1bdf-449a-4671-b640-97990dd0d405-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.938542 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.938464    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/790b1bdf-449a-4671-b640-97990dd0d405-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.938542 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.938498    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/790b1bdf-449a-4671-b640-97990dd0d405-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.938542 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.938537    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.938811 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.938610    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.938811 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.938646    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.938811 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.938670    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.938811 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.938778    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.939209 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.939179    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/790b1bdf-449a-4671-b640-97990dd0d405-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.939353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.939261    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.939513 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.939492    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.939689 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.939667    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.941513 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.941453    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/790b1bdf-449a-4671-b640-97990dd0d405-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.942067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.942042    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/790b1bdf-449a-4671-b640-97990dd0d405-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.946343 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.946268    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"]
Nov 02 22:15:24.996634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.996555    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/790b1bdf-449a-4671-b640-97990dd0d405-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:24.996776 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:24.996674    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h4s6\" (UniqueName: \"kubernetes.io/projected/790b1bdf-449a-4671-b640-97990dd0d405-kube-api-access-9h4s6\") pod \"maas-default-gateway-openshift-default-76f4876667-fzph7\" (UID: \"790b1bdf-449a-4671-b640-97990dd0d405\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:25.060420 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:25.059557    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:25.541173 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:25.541133    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"]
Nov 02 22:15:25.545157 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:15:25.545127    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod790b1bdf_449a_4671_b640_97990dd0d405.slice/crio-cd2c3d1938195528b9907b2288c3c6478ca8a9bb775ddbd6339800b579d9fef1 WatchSource:0}: Error finding container cd2c3d1938195528b9907b2288c3c6478ca8a9bb775ddbd6339800b579d9fef1: Status 404 returned error can't find the container with id cd2c3d1938195528b9907b2288c3c6478ca8a9bb775ddbd6339800b579d9fef1
Nov 02 22:15:25.548114 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:25.548084    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:15:25.548215 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:25.548164    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:15:25.548215 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:25.548210    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:15:25.776221 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:25.776172    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" event={"ID":"790b1bdf-449a-4671-b640-97990dd0d405","Type":"ContainerStarted","Data":"68748b2b313d3d6383b3d5cc745ecb8946787cdcc0a9cd74762253c61742981e"}
Nov 02 22:15:25.776462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:25.776249    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" event={"ID":"790b1bdf-449a-4671-b640-97990dd0d405","Type":"ContainerStarted","Data":"cd2c3d1938195528b9907b2288c3c6478ca8a9bb775ddbd6339800b579d9fef1"}
Nov 02 22:15:25.846129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:25.846037    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a" path="/var/lib/kubelet/pods/76b889a5-ed3c-4ef6-96d0-66b9c9d69e4a/volumes"
Nov 02 22:15:25.892130 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:25.892063    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" podStartSLOduration=1.892044123 podStartE2EDuration="1.892044123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:15:25.890010219 +0000 UTC m=+598.173819926" watchObservedRunningTime="2026-11-02 22:15:25.892044123 +0000 UTC m=+598.175853813"
Nov 02 22:15:26.061546 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:26.061507    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:26.063831 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:26.063796    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-fzph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 02 22:15:26.063975 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:26.063864    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" podUID="790b1bdf-449a-4671-b640-97990dd0d405" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 02 22:15:27.061936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:27.061895    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-fzph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 02 22:15:27.062357 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:27.061968    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" podUID="790b1bdf-449a-4671-b640-97990dd0d405" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 02 22:15:27.899859 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:27.899824    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:15:27.900396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:27.900340    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:15:28.064391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:28.061326    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-fzph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 02 22:15:28.064391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:28.061440    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" podUID="790b1bdf-449a-4671-b640-97990dd0d405" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 02 22:15:29.062033 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:29.061991    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-fzph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 02 22:15:29.062308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:29.062276    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" podUID="790b1bdf-449a-4671-b640-97990dd0d405" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 02 22:15:30.061482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:30.061439    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-fzph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 02 22:15:30.061902 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:30.061501    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" podUID="790b1bdf-449a-4671-b640-97990dd0d405" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 02 22:15:31.062389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:31.061825    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-fzph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 02 22:15:31.062389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:31.061894    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" podUID="790b1bdf-449a-4671-b640-97990dd0d405" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 02 22:15:32.062041 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:32.061997    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-fzph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 02 22:15:32.062289 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:32.062059    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" podUID="790b1bdf-449a-4671-b640-97990dd0d405" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 02 22:15:33.775000 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:33.774965    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:33.775537 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:33.775503    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:33.776396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:15:33.776356    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7"
Nov 02 22:15:37.575951 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:15:37.575911    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:16:07.575766 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:16:07.575725    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:16:37.083609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.083573    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"]
Nov 02 22:16:37.087244 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.087221    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.138556 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.138516    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-certs\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.138750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.138577    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4bc6e5d-e119-42e0-8254-df6772091d7e-istiod-ca-cert\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.138750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.138622    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-data\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.138750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.138676    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-token\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.138750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.138709    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlgcp\" (UniqueName: \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-kube-api-access-vlgcp\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.138969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.138768    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-podinfo\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.138969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.138803    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-credential-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.138969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.138831    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-envoy\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.138969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.138865    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.146712 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.146602    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-98a222-openshift-default-dockercfg-hhv4j\""
Nov 02 22:16:37.186026 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.185989    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"]
Nov 02 22:16:37.239681 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.239647    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-podinfo\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.239887 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.239699    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-credential-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.239887 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.239730    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-envoy\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.239887 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.239758    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.239887 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.239811    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-certs\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.239887 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.239840    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4bc6e5d-e119-42e0-8254-df6772091d7e-istiod-ca-cert\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.239887 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.239874    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-data\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.240180 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.239923    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-token\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.240180 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.239956    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlgcp\" (UniqueName: \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-kube-api-access-vlgcp\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.240746 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.240712    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.241200 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.241179    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-certs\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.242004 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.241982    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-data\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.242405 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.242385    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-credential-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.244059 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.244034    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4bc6e5d-e119-42e0-8254-df6772091d7e-istiod-ca-cert\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.247894 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.247861    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-podinfo\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.248913 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.248866    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-envoy\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.300052 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.300021    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlgcp\" (UniqueName: \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-kube-api-access-vlgcp\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.305809 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.305775    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-token\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.401195 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.401102    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:37.590548 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:16:37.590506    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:16:37.829467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.829436    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"]
Nov 02 22:16:37.832741 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:16:37.832713    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4bc6e5d_e119_42e0_8254_df6772091d7e.slice/crio-a09266e98dd9a80ebc1b73504515113a37a8ea91dd5721176d64e031ab5e701f WatchSource:0}: Error finding container a09266e98dd9a80ebc1b73504515113a37a8ea91dd5721176d64e031ab5e701f: Status 404 returned error can't find the container with id a09266e98dd9a80ebc1b73504515113a37a8ea91dd5721176d64e031ab5e701f
Nov 02 22:16:37.834951 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.834917    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:16:37.835031 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.834981    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:16:37.835031 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:37.835009    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:16:38.144758 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:38.144662    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" event={"ID":"f4bc6e5d-e119-42e0-8254-df6772091d7e","Type":"ContainerStarted","Data":"c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021"}
Nov 02 22:16:38.144758 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:38.144720    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" event={"ID":"f4bc6e5d-e119-42e0-8254-df6772091d7e","Type":"ContainerStarted","Data":"a09266e98dd9a80ebc1b73504515113a37a8ea91dd5721176d64e031ab5e701f"}
Nov 02 22:16:38.260396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:38.260329    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" podStartSLOduration=1.260313733 podStartE2EDuration="1.260313733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:16:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:16:38.258664439 +0000 UTC m=+670.542474137" watchObservedRunningTime="2026-11-02 22:16:38.260313733 +0000 UTC m=+670.544123419"
Nov 02 22:16:38.401639 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:38.401545    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:38.407198 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:38.407171    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:39.149692 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:39.149649    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:39.150637 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:39.150616    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:16:46.208353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.208320    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"]
Nov 02 22:16:46.211918 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.211901    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.260842 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.260811    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 22:16:46.261020 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.260941    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 22:16:46.307345 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.307308    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-49a557-serving-cert\""
Nov 02 22:16:46.307569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.307483    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-l4djg\""
Nov 02 22:16:46.307569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.307485    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 22:16:46.309151 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.309130    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"]
Nov 02 22:16:46.328092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.328062    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-maas-api-tls\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.328267 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.328148    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rqqj\" (UniqueName: \"kubernetes.io/projected/f3880123-ed80-4cec-8369-998133747622-kube-api-access-8rqqj\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.328267 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.328186    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3880123-ed80-4cec-8369-998133747622-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.328267 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.328253    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-metrics-certs\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.429157 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.429118    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-metrics-certs\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.429338 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.429200    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-maas-api-tls\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.429338 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.429264    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rqqj\" (UniqueName: \"kubernetes.io/projected/f3880123-ed80-4cec-8369-998133747622-kube-api-access-8rqqj\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.429338 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.429301    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3880123-ed80-4cec-8369-998133747622-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.429338 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:16:46.429313    5067 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-serving-cert: secret "maas-api-e2e-worker-w6-49a557-serving-cert" not found
Nov 02 22:16:46.429674 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:16:46.429430    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-maas-api-tls podName:f3880123-ed80-4cec-8369-998133747622 nodeName:}" failed. No retries permitted until 2026-11-02 22:16:46.929410266 +0000 UTC m=+679.213219950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-maas-api-tls") pod "maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp" (UID: "f3880123-ed80-4cec-8369-998133747622") : secret "maas-api-e2e-worker-w6-49a557-serving-cert" not found
Nov 02 22:16:46.430088 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.430065    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3880123-ed80-4cec-8369-998133747622-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.431904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.431881    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-metrics-certs\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.487729 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.487698    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rqqj\" (UniqueName: \"kubernetes.io/projected/f3880123-ed80-4cec-8369-998133747622-kube-api-access-8rqqj\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.934840 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.934746    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-maas-api-tls\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:46.937265 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:46.937237    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-maas-api-tls\") pod \"maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:47.125012 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:47.124974    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:47.498289 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:47.498255    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"]
Nov 02 22:16:47.502445 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:16:47.502417    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3880123_ed80_4cec_8369_998133747622.slice/crio-6a5f36c970ac1c2b66e0d8ac06669cff16113fc90f55a93220a1abfd03be0816 WatchSource:0}: Error finding container 6a5f36c970ac1c2b66e0d8ac06669cff16113fc90f55a93220a1abfd03be0816: Status 404 returned error can't find the container with id 6a5f36c970ac1c2b66e0d8ac06669cff16113fc90f55a93220a1abfd03be0816
Nov 02 22:16:47.503843 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:47.503825    5067 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:16:48.195756 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:48.195724    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp" event={"ID":"f3880123-ed80-4cec-8369-998133747622","Type":"ContainerStarted","Data":"6a5f36c970ac1c2b66e0d8ac06669cff16113fc90f55a93220a1abfd03be0816"}
Nov 02 22:16:49.201513 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:49.201474    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp" event={"ID":"f3880123-ed80-4cec-8369-998133747622","Type":"ContainerStarted","Data":"badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7"}
Nov 02 22:16:49.201939 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:49.201591    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:16:49.314161 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:49.314105    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp" podStartSLOduration=2.478468198 podStartE2EDuration="3.314090373s" totalImagesPullingTime="835.622175ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:16:49.313497769 +0000 UTC m=+681.597307475" watchObservedRunningTime="2026-11-02 22:16:49.314090373 +0000 UTC m=+681.597900060"
Nov 02 22:16:55.212252 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:16:55.212222    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:17:03.336506 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.336464    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"]
Nov 02 22:17:03.341510 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.341484    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:17:03.385386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.385141    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm2f9\" (UniqueName: \"kubernetes.io/projected/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-kube-api-access-zm2f9\") pod \"payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98\" (UID: \"0a7d1d82-d0e6-450d-b160-8bb80e78fa94\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:17:03.385386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.385265    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98\" (UID: \"0a7d1d82-d0e6-450d-b160-8bb80e78fa94\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:17:03.415032 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.414998    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-3224ea\""
Nov 02 22:17:03.415834 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.415776    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-3224ea-dockercfg-7swkf\""
Nov 02 22:17:03.419186 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.419162    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"]
Nov 02 22:17:03.485848 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.485808    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98\" (UID: \"0a7d1d82-d0e6-450d-b160-8bb80e78fa94\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:17:03.486022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.485905    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm2f9\" (UniqueName: \"kubernetes.io/projected/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-kube-api-access-zm2f9\") pod \"payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98\" (UID: \"0a7d1d82-d0e6-450d-b160-8bb80e78fa94\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:17:03.486692 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.486666    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98\" (UID: \"0a7d1d82-d0e6-450d-b160-8bb80e78fa94\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:17:03.543185 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.543154    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm2f9\" (UniqueName: \"kubernetes.io/projected/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-kube-api-access-zm2f9\") pod \"payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98\" (UID: \"0a7d1d82-d0e6-450d-b160-8bb80e78fa94\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:17:03.685425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:03.685338    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:17:04.057700 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:04.057668    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"]
Nov 02 22:17:04.061259 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:17:04.061230    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7d1d82_d0e6_450d_b160_8bb80e78fa94.slice/crio-6c1009138db9884e1a7123f8c66f9b20abdfc4547ea335776c2991c1e013dbb4 WatchSource:0}: Error finding container 6c1009138db9884e1a7123f8c66f9b20abdfc4547ea335776c2991c1e013dbb4: Status 404 returned error can't find the container with id 6c1009138db9884e1a7123f8c66f9b20abdfc4547ea335776c2991c1e013dbb4
Nov 02 22:17:04.285623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:04.285588    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98" event={"ID":"0a7d1d82-d0e6-450d-b160-8bb80e78fa94","Type":"ContainerStarted","Data":"6c1009138db9884e1a7123f8c66f9b20abdfc4547ea335776c2991c1e013dbb4"}
Nov 02 22:17:06.909531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:06.909495    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"]
Nov 02 22:17:06.912904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:06.912884    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:06.961726 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:06.961693    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-711bd4-serving-cert\""
Nov 02 22:17:07.012528 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.012495    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"]
Nov 02 22:17:07.016947 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.016920    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-maas-api-tls\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.017073 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.016955    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.017141 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.017083    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7tqd\" (UniqueName: \"kubernetes.io/projected/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-kube-api-access-p7tqd\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.017236 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.017220    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-metrics-certs\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.118316 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.118280    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-maas-api-tls\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.118316 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.118325    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.118597 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.118417    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7tqd\" (UniqueName: \"kubernetes.io/projected/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-kube-api-access-p7tqd\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.118597 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:17:07.118477    5067 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-serving-cert: secret "maas-api-e2e-worker-w0-711bd4-serving-cert" not found
Nov 02 22:17:07.118597 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:17:07.118554    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-maas-api-tls podName:c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1 nodeName:}" failed. No retries permitted until 2026-11-02 22:17:07.618523223 +0000 UTC m=+699.902332906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-maas-api-tls") pod "maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5" (UID: "c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1") : secret "maas-api-e2e-worker-w0-711bd4-serving-cert" not found
Nov 02 22:17:07.118597 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.118569    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-metrics-certs\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.119215 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.119184    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.121178 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.121157    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-metrics-certs\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.178200 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.178030    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7tqd\" (UniqueName: \"kubernetes.io/projected/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-kube-api-access-p7tqd\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.303996 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.303955    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98" event={"ID":"0a7d1d82-d0e6-450d-b160-8bb80e78fa94","Type":"ContainerStarted","Data":"e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44"}
Nov 02 22:17:07.304188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.304105    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:17:07.413670 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.413609    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98" podStartSLOduration=1.5396404129999999 podStartE2EDuration="4.413595493s" totalImagesPullingTime="2.87395508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:17:07.413546311 +0000 UTC m=+699.697356019" watchObservedRunningTime="2026-11-02 22:17:07.413595493 +0000 UTC m=+699.697405179"
Nov 02 22:17:07.582042 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:17:07.581999    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:17:07.623171 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.623135    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-maas-api-tls\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.625686 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.625661    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-maas-api-tls\") pod \"maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:07.825527 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:07.825489    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:08.246269 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:08.246246    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"]
Nov 02 22:17:08.247093 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:17:08.247071    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23b9c5e_72b9_4b4e_9cfd_21f7a01796b1.slice/crio-779a6127e0b374c89da813ae7be1f92a2548d5f4b948dfdc57adc5e9a8958f32 WatchSource:0}: Error finding container 779a6127e0b374c89da813ae7be1f92a2548d5f4b948dfdc57adc5e9a8958f32: Status 404 returned error can't find the container with id 779a6127e0b374c89da813ae7be1f92a2548d5f4b948dfdc57adc5e9a8958f32
Nov 02 22:17:08.309438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:08.309396    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5" event={"ID":"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1","Type":"ContainerStarted","Data":"779a6127e0b374c89da813ae7be1f92a2548d5f4b948dfdc57adc5e9a8958f32"}
Nov 02 22:17:09.315913 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:09.315882    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5" event={"ID":"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1","Type":"ContainerStarted","Data":"f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3"}
Nov 02 22:17:09.316337 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:09.316019    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:09.447569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:09.447461    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5" podStartSLOduration=2.519225056 podStartE2EDuration="3.447443282s" totalImagesPullingTime="928.218226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:17:09.446786972 +0000 UTC m=+701.730596658" watchObservedRunningTime="2026-11-02 22:17:09.447443282 +0000 UTC m=+701.731252968"
Nov 02 22:17:11.886525 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:11.886493    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"]
Nov 02 22:17:11.890249 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:11.890224    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:11.939703 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:11.939672    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-8263a4-serving-cert\""
Nov 02 22:17:11.971962 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:11.971912    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:11.972315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:11.972285    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz4kc\" (UniqueName: \"kubernetes.io/projected/92b04e93-2dfc-4b66-868e-a4cebe5722e5-kube-api-access-lz4kc\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:11.972474 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:11.972341    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-metrics-certs\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:11.972474 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:11.972408    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92b04e93-2dfc-4b66-868e-a4cebe5722e5-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:11.987706 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:11.987670    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"]
Nov 02 22:17:12.073672 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:12.073634    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:12.073897 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:12.073724    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz4kc\" (UniqueName: \"kubernetes.io/projected/92b04e93-2dfc-4b66-868e-a4cebe5722e5-kube-api-access-lz4kc\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:12.073897 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:12.073754    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-metrics-certs\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:12.073897 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:12.073788    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92b04e93-2dfc-4b66-868e-a4cebe5722e5-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:12.073897 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:17:12.073802    5067 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-serving-cert: secret "maas-api-e2e-worker-w2-8263a4-serving-cert" not found
Nov 02 22:17:12.073897 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:17:12.073884    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-maas-api-tls podName:92b04e93-2dfc-4b66-868e-a4cebe5722e5 nodeName:}" failed. No retries permitted until 2026-11-02 22:17:12.573864354 +0000 UTC m=+704.857674049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-maas-api-tls") pod "maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl" (UID: "92b04e93-2dfc-4b66-868e-a4cebe5722e5") : secret "maas-api-e2e-worker-w2-8263a4-serving-cert" not found
Nov 02 22:17:12.074971 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:12.074943    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92b04e93-2dfc-4b66-868e-a4cebe5722e5-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:12.084623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:12.084569    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-metrics-certs\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:12.134068 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:12.134038    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz4kc\" (UniqueName: \"kubernetes.io/projected/92b04e93-2dfc-4b66-868e-a4cebe5722e5-kube-api-access-lz4kc\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:12.579831 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:12.579787    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:12.582658 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:12.582620    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:12.804067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:12.804026    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:13.193124 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:13.193038    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"]
Nov 02 22:17:13.197791 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:17:13.197749    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b04e93_2dfc_4b66_868e_a4cebe5722e5.slice/crio-5df3452295be86144cfb921daf7141fbf8fb2019e6dce21801d44c9fa0718be7 WatchSource:0}: Error finding container 5df3452295be86144cfb921daf7141fbf8fb2019e6dce21801d44c9fa0718be7: Status 404 returned error can't find the container with id 5df3452295be86144cfb921daf7141fbf8fb2019e6dce21801d44c9fa0718be7
Nov 02 22:17:13.338084 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:13.338036    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl" event={"ID":"92b04e93-2dfc-4b66-868e-a4cebe5722e5","Type":"ContainerStarted","Data":"5df3452295be86144cfb921daf7141fbf8fb2019e6dce21801d44c9fa0718be7"}
Nov 02 22:17:14.344986 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:14.344949    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl" event={"ID":"92b04e93-2dfc-4b66-868e-a4cebe5722e5","Type":"ContainerStarted","Data":"edb77b9e14a2a4883eb047a640ffdc502bab189cfea833bd7f20a404104a8c44"}
Nov 02 22:17:14.345443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:14.345069    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:14.477713 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:14.477651    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl" podStartSLOduration=2.468654687 podStartE2EDuration="3.477632335s" totalImagesPullingTime="1.008977648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:17:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:17:14.477190265 +0000 UTC m=+706.761000026" watchObservedRunningTime="2026-11-02 22:17:14.477632335 +0000 UTC m=+706.761442023"
Nov 02 22:17:15.326534 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:15.326502    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:17:18.312723 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:18.312685    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:17:20.355964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:20.355929    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:17:29.704517 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.704480    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"]
Nov 02 22:17:29.711431 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.711410    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.808714 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.808675    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-49a557\"/\"kube-root-ca.crt\""
Nov 02 22:17:29.808906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.808835    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-49a557\"/\"default-dockercfg-bzn49\""
Nov 02 22:17:29.808906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.808842    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-49a557\"/\"simulator-w6-49a557-kserve-self-signed-certs\""
Nov 02 22:17:29.808906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.808876    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-49a557\"/\"openshift-service-ca.crt\""
Nov 02 22:17:29.810754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.810730    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"]
Nov 02 22:17:29.856592 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.856555    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-tmp-dir\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.856752 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.856606    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28414ade-e581-4334-9bd9-0d85a4aa33c6-tls-certs\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.856752 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.856684    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-home\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.856859 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.856790    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7zr7\" (UniqueName: \"kubernetes.io/projected/28414ade-e581-4334-9bd9-0d85a4aa33c6-kube-api-access-g7zr7\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.856902 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.856857    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-dshm\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.856902 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.856895    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-model-cache\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.958533 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.958426    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-dshm\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.958726 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.958617    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-model-cache\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.958916 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.958889    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-tmp-dir\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.959120 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.959096    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-model-cache\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.959272 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.959138    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28414ade-e581-4334-9bd9-0d85a4aa33c6-tls-certs\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.959272 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.959173    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-home\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.959445 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.959269    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-tmp-dir\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.960188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.960162    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7zr7\" (UniqueName: \"kubernetes.io/projected/28414ade-e581-4334-9bd9-0d85a4aa33c6-kube-api-access-g7zr7\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.960702 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.960677    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-home\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.961131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.961102    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-dshm\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:29.962347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:29.962327    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28414ade-e581-4334-9bd9-0d85a4aa33c6-tls-certs\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:30.018428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:30.018392    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7zr7\" (UniqueName: \"kubernetes.io/projected/28414ade-e581-4334-9bd9-0d85a4aa33c6-kube-api-access-g7zr7\") pod \"simulator-w6-49a557-kserve-67589c998-zf4dh\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") " pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:30.023091 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:30.023061    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:30.392638 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:30.392615    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"]
Nov 02 22:17:30.395338 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:17:30.395314    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28414ade_e581_4334_9bd9_0d85a4aa33c6.slice/crio-eacbb0e32f4395257f27c6180e61e434eb91587a1fe45c4a750655a10cf2c832 WatchSource:0}: Error finding container eacbb0e32f4395257f27c6180e61e434eb91587a1fe45c4a750655a10cf2c832: Status 404 returned error can't find the container with id eacbb0e32f4395257f27c6180e61e434eb91587a1fe45c4a750655a10cf2c832
Nov 02 22:17:30.440823 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:30.440784    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" event={"ID":"28414ade-e581-4334-9bd9-0d85a4aa33c6","Type":"ContainerStarted","Data":"eacbb0e32f4395257f27c6180e61e434eb91587a1fe45c4a750655a10cf2c832"}
Nov 02 22:17:33.475052 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:33.475009    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" event={"ID":"28414ade-e581-4334-9bd9-0d85a4aa33c6","Type":"ContainerStarted","Data":"26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0"}
Nov 02 22:17:33.586777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:33.586718    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podStartSLOduration=2.115258166 podStartE2EDuration="4.586703048s" totalImagesPullingTime="2.471444882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:17:33.585570559 +0000 UTC m=+725.869380247" watchObservedRunningTime="2026-11-02 22:17:33.586703048 +0000 UTC m=+725.870512736"
Nov 02 22:17:34.023579 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:34.023548    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:37.583692 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:17:37.583647    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:17:38.692035 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.691995    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"]
Nov 02 22:17:38.696588 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.696567    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:17:38.762092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.762053    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"]
Nov 02 22:17:38.773437 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.773401    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz8cr\" (UniqueName: \"kubernetes.io/projected/782626e4-258f-4569-bd26-9741d46f7909-kube-api-access-zz8cr\") pod \"payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh\" (UID: \"782626e4-258f-4569-bd26-9741d46f7909\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:17:38.773653 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.773549    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/782626e4-258f-4569-bd26-9741d46f7909-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh\" (UID: \"782626e4-258f-4569-bd26-9741d46f7909\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:17:38.805151 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.805115    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-ca85db\""
Nov 02 22:17:38.805151 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.805115    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-ca85db-dockercfg-md5gm\""
Nov 02 22:17:38.874329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.874292    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/782626e4-258f-4569-bd26-9741d46f7909-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh\" (UID: \"782626e4-258f-4569-bd26-9741d46f7909\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:17:38.874558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.874409    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz8cr\" (UniqueName: \"kubernetes.io/projected/782626e4-258f-4569-bd26-9741d46f7909-kube-api-access-zz8cr\") pod \"payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh\" (UID: \"782626e4-258f-4569-bd26-9741d46f7909\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:17:38.875164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.875138    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/782626e4-258f-4569-bd26-9741d46f7909-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh\" (UID: \"782626e4-258f-4569-bd26-9741d46f7909\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:17:38.932500 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:38.932467    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz8cr\" (UniqueName: \"kubernetes.io/projected/782626e4-258f-4569-bd26-9741d46f7909-kube-api-access-zz8cr\") pod \"payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh\" (UID: \"782626e4-258f-4569-bd26-9741d46f7909\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:17:39.009322 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:39.009282    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:17:39.380873 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:39.380789    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"]
Nov 02 22:17:39.384057 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:17:39.384023    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod782626e4_258f_4569_bd26_9741d46f7909.slice/crio-436d75ebba5254cf18fd104ac6d87ec4fd43c478aca2cbdb339718818eddbf8e WatchSource:0}: Error finding container 436d75ebba5254cf18fd104ac6d87ec4fd43c478aca2cbdb339718818eddbf8e: Status 404 returned error can't find the container with id 436d75ebba5254cf18fd104ac6d87ec4fd43c478aca2cbdb339718818eddbf8e
Nov 02 22:17:39.510112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:39.510076    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh" event={"ID":"782626e4-258f-4569-bd26-9741d46f7909","Type":"ContainerStarted","Data":"9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b"}
Nov 02 22:17:39.510284 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:39.510139    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh" event={"ID":"782626e4-258f-4569-bd26-9741d46f7909","Type":"ContainerStarted","Data":"436d75ebba5254cf18fd104ac6d87ec4fd43c478aca2cbdb339718818eddbf8e"}
Nov 02 22:17:39.510284 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:39.510169    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:17:39.620471 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:39.620085    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh" podStartSLOduration=1.620065176 podStartE2EDuration="1.620065176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:17:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:17:39.618966377 +0000 UTC m=+731.902776081" watchObservedRunningTime="2026-11-02 22:17:39.620065176 +0000 UTC m=+731.903874864"
Nov 02 22:17:40.023311 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:40.023267    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:40.040840 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:40.040809    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:40.191570 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:40.191524    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:17:40.191826 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:40.191604    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:17:40.528805 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:40.528770    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:17:42.528447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:42.528343    5067 generic.go:356] "Generic (PLEG): container finished" podID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerID="c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021" exitCode=0
Nov 02 22:17:42.528447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:42.528424    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" event={"ID":"f4bc6e5d-e119-42e0-8254-df6772091d7e","Type":"ContainerDied","Data":"c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021"}
Nov 02 22:17:42.528961 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:42.528894    5067 scope.go:122] "RemoveContainer" containerID="c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021"
Nov 02 22:17:42.530564 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:42.530528    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:17:42.530647 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:42.530629    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:17:42.530707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:42.530686    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:17:43.549392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:43.548689    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" event={"ID":"f4bc6e5d-e119-42e0-8254-df6772091d7e","Type":"ContainerStarted","Data":"b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5"}
Nov 02 22:17:44.402295 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:44.402255    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:17:44.402621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:44.402601    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 02 22:17:44.402674 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:44.402645    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 02 22:17:45.401701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:45.401666    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 02 22:17:45.402121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:45.401719    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 02 22:17:46.402489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:46.402457    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 02 22:17:46.402933 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:46.402511    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 02 22:17:47.402270 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:47.402234    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 02 22:17:47.402471 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:47.402286    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 02 22:17:48.402381 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:48.402331    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 02 22:17:48.402781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:48.402404    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 02 22:17:49.405471 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:49.405438    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:17:49.405886 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:49.405702    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:17:49.406431 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:49.406413    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:17:50.519110 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:50.519079    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:17:56.328407 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.328346    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"]
Nov 02 22:17:56.331840 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.331820    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.381424 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.381398    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-fe810f\"/\"openshift-service-ca.crt\""
Nov 02 22:17:56.427345 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.427312    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-fe810f\"/\"kube-root-ca.crt\""
Nov 02 22:17:56.427576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.427556    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-fe810f\"/\"simulator-w5-fe810f-kserve-self-signed-certs\""
Nov 02 22:17:56.427688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.427644    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-fe810f\"/\"default-dockercfg-svmfr\""
Nov 02 22:17:56.429778 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.429749    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"]
Nov 02 22:17:56.476736 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.476698    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7q8b\" (UniqueName: \"kubernetes.io/projected/af6506ab-99cf-4bc1-89b8-3340e6f0c411-kube-api-access-t7q8b\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.476922 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.476788    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-home\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.476922 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.476843    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-dshm\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.476922 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.476872    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tmp-dir\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.477116 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.476971    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-model-cache\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.477216 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.477157    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tls-certs\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.578039 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.578005    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tls-certs\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.578246 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.578059    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7q8b\" (UniqueName: \"kubernetes.io/projected/af6506ab-99cf-4bc1-89b8-3340e6f0c411-kube-api-access-t7q8b\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.578246 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.578115    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-home\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.578246 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.578171    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-dshm\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.578246 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.578193    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tmp-dir\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.578523 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.578250    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-model-cache\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.578669 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.578644    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tmp-dir\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.578807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.578680    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-model-cache\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.578807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.578734    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-home\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.580533 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.580503    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-dshm\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.580888 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.580868    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tls-certs\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.636186 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.636156    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7q8b\" (UniqueName: \"kubernetes.io/projected/af6506ab-99cf-4bc1-89b8-3340e6f0c411-kube-api-access-t7q8b\") pod \"simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") " pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:56.642983 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:56.642957    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:57.014263 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:57.014236    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"]
Nov 02 22:17:57.016803 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:17:57.016775    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf6506ab_99cf_4bc1_89b8_3340e6f0c411.slice/crio-abfa18a8b6f05913340087f9bf81ecfe3d11fb46501fa8ed6b9021376956f3bc WatchSource:0}: Error finding container abfa18a8b6f05913340087f9bf81ecfe3d11fb46501fa8ed6b9021376956f3bc: Status 404 returned error can't find the container with id abfa18a8b6f05913340087f9bf81ecfe3d11fb46501fa8ed6b9021376956f3bc
Nov 02 22:17:57.632919 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:57.632880    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" event={"ID":"af6506ab-99cf-4bc1-89b8-3340e6f0c411","Type":"ContainerStarted","Data":"636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23"}
Nov 02 22:17:57.633414 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:57.632933    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" event={"ID":"af6506ab-99cf-4bc1-89b8-3340e6f0c411","Type":"ContainerStarted","Data":"abfa18a8b6f05913340087f9bf81ecfe3d11fb46501fa8ed6b9021376956f3bc"}
Nov 02 22:17:57.643784 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:57.643759    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:17:57.744235 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:17:57.744181    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podStartSLOduration=1.744161648 podStartE2EDuration="1.744161648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:17:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:17:57.742924209 +0000 UTC m=+750.026733896" watchObservedRunningTime="2026-11-02 22:17:57.744161648 +0000 UTC m=+750.027971335"
Nov 02 22:18:01.159737 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.159697    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"]
Nov 02 22:18:01.162977 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.162954    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.212425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.212348    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3224ea\"/\"openshift-service-ca.crt\""
Nov 02 22:18:01.225633 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.225598    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-model-cache\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.225821 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.225647    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tls-certs\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.225821 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.225673    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-dshm\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.225821 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.225760    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-home\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.225821 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.225801    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq4dw\" (UniqueName: \"kubernetes.io/projected/b16554d0-6330-4ffe-89a4-5c91509a3f0e-kube-api-access-tq4dw\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.226104 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.225828    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tmp-dir\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.258665 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.258631    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3224ea\"/\"kube-root-ca.crt\""
Nov 02 22:18:01.258846 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.258766    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3224ea\"/\"default-dockercfg-twqdc\""
Nov 02 22:18:01.258916 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.258830    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3224ea\"/\"simulator-w4-3224ea-kserve-self-signed-certs\""
Nov 02 22:18:01.262199 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.262161    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"]
Nov 02 22:18:01.326839 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.326807    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-model-cache\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.327046 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.326859    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tls-certs\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.327046 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.326882    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-dshm\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.327046 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.326912    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-home\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.327046 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.326946    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq4dw\" (UniqueName: \"kubernetes.io/projected/b16554d0-6330-4ffe-89a4-5c91509a3f0e-kube-api-access-tq4dw\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.327046 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.326966    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tmp-dir\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.327317 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.327268    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-model-cache\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.327391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.327315    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-home\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.327391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.327338    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tmp-dir\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.329182 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.329160    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-dshm\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.329611 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.329593    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tls-certs\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.385318 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.385280    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq4dw\" (UniqueName: \"kubernetes.io/projected/b16554d0-6330-4ffe-89a4-5c91509a3f0e-kube-api-access-tq4dw\") pod \"simulator-w4-3224ea-kserve-7ccc744d68-85s74\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") " pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.478447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.478404    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:01.856578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:01.856549    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"]
Nov 02 22:18:01.858161 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:18:01.858130    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb16554d0_6330_4ffe_89a4_5c91509a3f0e.slice/crio-b5822346eebb673ab644f02d74aee7011ca2a4b426048e03458879286492d823 WatchSource:0}: Error finding container b5822346eebb673ab644f02d74aee7011ca2a4b426048e03458879286492d823: Status 404 returned error can't find the container with id b5822346eebb673ab644f02d74aee7011ca2a4b426048e03458879286492d823
Nov 02 22:18:02.667506 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:02.667469    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" event={"ID":"b16554d0-6330-4ffe-89a4-5c91509a3f0e","Type":"ContainerStarted","Data":"381e0660c4513ad43df7a78785b077238642b1ce160136d815512828126490e4"}
Nov 02 22:18:02.667506 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:02.667513    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" event={"ID":"b16554d0-6330-4ffe-89a4-5c91509a3f0e","Type":"ContainerStarted","Data":"b5822346eebb673ab644f02d74aee7011ca2a4b426048e03458879286492d823"}
Nov 02 22:18:02.778931 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:02.778875    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podStartSLOduration=1.778863388 podStartE2EDuration="1.778863388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:18:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:18:02.777473532 +0000 UTC m=+755.061283238" watchObservedRunningTime="2026-11-02 22:18:02.778863388 +0000 UTC m=+755.062673074"
Nov 02 22:18:03.478512 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:03.478467    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:06.643239 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:06.643197    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:18:06.656131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:06.656103    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:18:06.698349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:06.698321    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:18:07.575714 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:18:07.575673    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:18:11.478904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:11.478868    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:11.491360 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:11.491331    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:11.725737 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:11.725704    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:18:19.200630 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.200590    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"]
Nov 02 22:18:19.203805 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.203784    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.210225 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.210192    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-home\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.210388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.210309    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-model-cache\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.210388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.210344    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr8pn\" (UniqueName: \"kubernetes.io/projected/e83f7463-2538-40cb-a284-ee2179b6d690-kube-api-access-tr8pn\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.210508 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.210407    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-dshm\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.210508 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.210466    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83f7463-2538-40cb-a284-ee2179b6d690-tls-certs\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.210645 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.210629    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-tmp-dir\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.299230 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.299196    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-711bd4\"/\"openshift-service-ca.crt\""
Nov 02 22:18:19.299230 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.299223    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-711bd4\"/\"kube-root-ca.crt\""
Nov 02 22:18:19.299511 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.299298    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-711bd4\"/\"default-dockercfg-46b9k\""
Nov 02 22:18:19.299511 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.299481    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-711bd4\"/\"simulator-w0-711bd4-kserve-self-signed-certs\""
Nov 02 22:18:19.301597 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.301580    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"]
Nov 02 22:18:19.312014 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.311993    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-model-cache\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.312144 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.312019    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr8pn\" (UniqueName: \"kubernetes.io/projected/e83f7463-2538-40cb-a284-ee2179b6d690-kube-api-access-tr8pn\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.312144 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.312041    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-dshm\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.312254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.312169    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83f7463-2538-40cb-a284-ee2179b6d690-tls-certs\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.312254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.312245    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-tmp-dir\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.312720 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.312299    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-home\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.312720 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.312462    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-model-cache\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.312720 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.312635    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-home\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.312720 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.312653    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-tmp-dir\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.314305 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.314284    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-dshm\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.314636 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.314620    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83f7463-2538-40cb-a284-ee2179b6d690-tls-certs\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.370332 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.370299    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr8pn\" (UniqueName: \"kubernetes.io/projected/e83f7463-2538-40cb-a284-ee2179b6d690-kube-api-access-tr8pn\") pod \"simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") " pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.516474 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.516435    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:19.893634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:19.893506    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"]
Nov 02 22:18:19.896359 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:18:19.896333    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode83f7463_2538_40cb_a284_ee2179b6d690.slice/crio-67d7f0e6da5e5bdda3743e420881bdaff2f9a30b6c576ed21f2b61fc9b5f8d66 WatchSource:0}: Error finding container 67d7f0e6da5e5bdda3743e420881bdaff2f9a30b6c576ed21f2b61fc9b5f8d66: Status 404 returned error can't find the container with id 67d7f0e6da5e5bdda3743e420881bdaff2f9a30b6c576ed21f2b61fc9b5f8d66
Nov 02 22:18:20.764892 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:20.764851    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" event={"ID":"e83f7463-2538-40cb-a284-ee2179b6d690","Type":"ContainerStarted","Data":"a025d17782de4732d4cb9150a5a3ff3b2ce46f611de207df641583e86a51c297"}
Nov 02 22:18:20.765424 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:20.764899    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" event={"ID":"e83f7463-2538-40cb-a284-ee2179b6d690","Type":"ContainerStarted","Data":"67d7f0e6da5e5bdda3743e420881bdaff2f9a30b6c576ed21f2b61fc9b5f8d66"}
Nov 02 22:18:20.876058 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:20.876000    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podStartSLOduration=1.875985478 podStartE2EDuration="1.875985478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:18:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:18:20.87495231 +0000 UTC m=+773.158762024" watchObservedRunningTime="2026-11-02 22:18:20.875985478 +0000 UTC m=+773.159795221"
Nov 02 22:18:21.516548 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:21.516508    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:27.935318 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:27.935285    5067 scope.go:122] "RemoveContainer" containerID="0565ebf007c85638fb28a83344706cf944800077156b743f9a97eac9a7a6fe14"
Nov 02 22:18:29.516677 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:29.516633    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:29.529466 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:29.529438    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:29.830355 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:29.830282    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:18:32.358792 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.358755    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"]
Nov 02 22:18:32.362598 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.362577    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.449837 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.449803    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-home\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.449837 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.449849    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-tmp-dir\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.450054 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.449888    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-dshm\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.450054 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.449970    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-model-cache\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.450054 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.449987    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ead096-3c6c-487e-84a1-0803bafc984e-tls-certs\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.450054 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.450013    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crgvc\" (UniqueName: \"kubernetes.io/projected/a8ead096-3c6c-487e-84a1-0803bafc984e-kube-api-access-crgvc\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.457953 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.457913    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-8263a4\"/\"openshift-service-ca.crt\""
Nov 02 22:18:32.457953 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.457922    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-8263a4\"/\"default-dockercfg-bj2sh\""
Nov 02 22:18:32.457953 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.457954    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-8263a4\"/\"simulator-w2-8263a4-kserve-self-signed-certs\""
Nov 02 22:18:32.458217 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.458111    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-8263a4\"/\"kube-root-ca.crt\""
Nov 02 22:18:32.460142 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.460120    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"]
Nov 02 22:18:32.551146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.551111    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-dshm\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.551325 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.551182    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-model-cache\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.551325 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.551210    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ead096-3c6c-487e-84a1-0803bafc984e-tls-certs\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.551325 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.551248    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crgvc\" (UniqueName: \"kubernetes.io/projected/a8ead096-3c6c-487e-84a1-0803bafc984e-kube-api-access-crgvc\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.551531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.551346    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-home\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.551531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.551428    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-tmp-dir\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.551909 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.551685    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-model-cache\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.551909 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.551734    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-home\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.551909 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.551811    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-tmp-dir\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.553563 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.553540    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-dshm\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.554039 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.554024    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ead096-3c6c-487e-84a1-0803bafc984e-tls-certs\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.609354 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.609278    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crgvc\" (UniqueName: \"kubernetes.io/projected/a8ead096-3c6c-487e-84a1-0803bafc984e-kube-api-access-crgvc\") pod \"simulator-w2-8263a4-kserve-68cc7c8765-6k6rm\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") " pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:32.673519 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:32.673468    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:33.059817 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:33.059669    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"]
Nov 02 22:18:33.062590 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:18:33.062561    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8ead096_3c6c_487e_84a1_0803bafc984e.slice/crio-c83027aff65bb335ad4077a51a9203596f75797d13aeb02345c7aba8ab748c8f WatchSource:0}: Error finding container c83027aff65bb335ad4077a51a9203596f75797d13aeb02345c7aba8ab748c8f: Status 404 returned error can't find the container with id c83027aff65bb335ad4077a51a9203596f75797d13aeb02345c7aba8ab748c8f
Nov 02 22:18:33.846473 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:33.846442    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" event={"ID":"a8ead096-3c6c-487e-84a1-0803bafc984e","Type":"ContainerStarted","Data":"f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4"}
Nov 02 22:18:33.846863 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:33.846488    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" event={"ID":"a8ead096-3c6c-487e-84a1-0803bafc984e","Type":"ContainerStarted","Data":"c83027aff65bb335ad4077a51a9203596f75797d13aeb02345c7aba8ab748c8f"}
Nov 02 22:18:33.956697 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:33.956630    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podStartSLOduration=1.956616672 podStartE2EDuration="1.956616672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:18:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:18:33.954969771 +0000 UTC m=+786.238779459" watchObservedRunningTime="2026-11-02 22:18:33.956616672 +0000 UTC m=+786.240426429"
Nov 02 22:18:34.673641 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:34.673601    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:37.577142 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:18:37.577102    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:18:42.674063 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:42.674023    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:42.690093 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:42.690065    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:42.906506 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:42.906469    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:18:58.577549 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.577511    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"]
Nov 02 22:18:58.580464 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.580446    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.625038 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.625006    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-home\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.625038 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.625044    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg2mn\" (UniqueName: \"kubernetes.io/projected/40a9abc6-a069-44c5-b62a-b700c8994560-kube-api-access-cg2mn\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.625293 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.625085    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a9abc6-a069-44c5-b62a-b700c8994560-tls-certs\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.625293 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.625226    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-tmp-dir\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.625427 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.625289    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-dshm\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.625483 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.625439    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-model-cache\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.675968 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.675932    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-fe810f\"/\"premium-w5-fe810f-kserve-self-signed-certs\""
Nov 02 22:18:58.678086 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.678064    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"]
Nov 02 22:18:58.726997 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.726960    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-model-cache\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.727201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.727036    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-home\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.727201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.727142    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg2mn\" (UniqueName: \"kubernetes.io/projected/40a9abc6-a069-44c5-b62a-b700c8994560-kube-api-access-cg2mn\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.727334 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.727198    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a9abc6-a069-44c5-b62a-b700c8994560-tls-certs\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.727334 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.727263    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-tmp-dir\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.728191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.727473    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-dshm\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.728191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.727514    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-home\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.728191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.728129    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-tmp-dir\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.728843 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.728823    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-model-cache\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.730469 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.730442    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-dshm\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.731142 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.731118    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a9abc6-a069-44c5-b62a-b700c8994560-tls-certs\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.785847 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.785812    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg2mn\" (UniqueName: \"kubernetes.io/projected/40a9abc6-a069-44c5-b62a-b700c8994560-kube-api-access-cg2mn\") pod \"premium-w5-fe810f-kserve-84cf95ff65-4jb94\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") " pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:58.892413 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:58.892298    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:18:59.290484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:59.290456    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"]
Nov 02 22:18:59.293000 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:18:59.292971    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40a9abc6_a069_44c5_b62a_b700c8994560.slice/crio-dd7be79d27ee5fc22934eaa1917dced336d13b85eb354c644e3897ebdde53998 WatchSource:0}: Error finding container dd7be79d27ee5fc22934eaa1917dced336d13b85eb354c644e3897ebdde53998: Status 404 returned error can't find the container with id dd7be79d27ee5fc22934eaa1917dced336d13b85eb354c644e3897ebdde53998
Nov 02 22:18:59.992316 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:59.992279    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" event={"ID":"40a9abc6-a069-44c5-b62a-b700c8994560","Type":"ContainerStarted","Data":"aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39"}
Nov 02 22:18:59.992316 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:18:59.992328    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" event={"ID":"40a9abc6-a069-44c5-b62a-b700c8994560","Type":"ContainerStarted","Data":"dd7be79d27ee5fc22934eaa1917dced336d13b85eb354c644e3897ebdde53998"}
Nov 02 22:19:00.106049 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:00.105991    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podStartSLOduration=2.105974165 podStartE2EDuration="2.105974165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:18:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:19:00.103599288 +0000 UTC m=+812.387409088" watchObservedRunningTime="2026-11-02 22:19:00.105974165 +0000 UTC m=+812.389783851"
Nov 02 22:19:00.892792 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:00.892750    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:19:07.575522 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:19:07.575483    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:19:08.892801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:08.892756    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:19:08.905353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:08.905325    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:19:09.060739 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:09.060698    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:19:31.066353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.066324    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"]
Nov 02 22:19:31.069352 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.069331    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.164739 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.164704    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-711bd4\"/\"premium-w0-711bd4-kserve-self-signed-certs\""
Nov 02 22:19:31.164951 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.164925    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02407f93-fd96-49c9-a18c-348651482119-tls-certs\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.165021 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.164961    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-model-cache\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.165094 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.165041    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-dshm\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.165094 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.165078    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-tmp-dir\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.165198 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.165112    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-home\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.165250 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.165231    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7cr2\" (UniqueName: \"kubernetes.io/projected/02407f93-fd96-49c9-a18c-348651482119-kube-api-access-k7cr2\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.167197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.167175    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"]
Nov 02 22:19:31.266056 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.266023    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02407f93-fd96-49c9-a18c-348651482119-tls-certs\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.266056 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.266058    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-model-cache\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.266299 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.266098    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-dshm\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.266299 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.266226    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-tmp-dir\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.266428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.266301    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-home\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.266428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.266415    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7cr2\" (UniqueName: \"kubernetes.io/projected/02407f93-fd96-49c9-a18c-348651482119-kube-api-access-k7cr2\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.266561 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.266532    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-model-cache\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.266662 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.266641    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-home\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.266926 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.266907    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-tmp-dir\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.268509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.268482    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-dshm\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.268798 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.268777    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02407f93-fd96-49c9-a18c-348651482119-tls-certs\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.326703 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.326635    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7cr2\" (UniqueName: \"kubernetes.io/projected/02407f93-fd96-49c9-a18c-348651482119-kube-api-access-k7cr2\") pod \"premium-w0-711bd4-kserve-7bb5f5c567-dpwds\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") " pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.381969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.381944    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:31.777223 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:31.777197    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"]
Nov 02 22:19:31.779840 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:19:31.779807    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02407f93_fd96_49c9_a18c_348651482119.slice/crio-1b30d230e37ee2ea6be94357bbcd8113ca7b71abc68b713a91ae1997af9be4ea WatchSource:0}: Error finding container 1b30d230e37ee2ea6be94357bbcd8113ca7b71abc68b713a91ae1997af9be4ea: Status 404 returned error can't find the container with id 1b30d230e37ee2ea6be94357bbcd8113ca7b71abc68b713a91ae1997af9be4ea
Nov 02 22:19:32.186238 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:32.186143    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" event={"ID":"02407f93-fd96-49c9-a18c-348651482119","Type":"ContainerStarted","Data":"bdcd1a4cbb8ebd728f2ddee03b7942e7ee4a9636527bf14f1fd375719ed3ad6f"}
Nov 02 22:19:32.186238 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:32.186195    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" event={"ID":"02407f93-fd96-49c9-a18c-348651482119","Type":"ContainerStarted","Data":"1b30d230e37ee2ea6be94357bbcd8113ca7b71abc68b713a91ae1997af9be4ea"}
Nov 02 22:19:32.298128 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:32.298076    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podStartSLOduration=1.2980602700000001 podStartE2EDuration="1.29806027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:19:32.295761007 +0000 UTC m=+844.579570696" watchObservedRunningTime="2026-11-02 22:19:32.29806027 +0000 UTC m=+844.581869935"
Nov 02 22:19:32.382817 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:32.382781    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:37.577720 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:19:37.577671    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:19:41.382849 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:41.382811    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:41.399453 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:41.399419    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:19:42.247733 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:19:42.247702    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:20:07.580472 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:20:07.580431    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:20:27.942957 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:27.942925    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:20:27.946833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:27.946808    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:20:34.992536 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:34.992499    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"]
Nov 02 22:20:34.995788 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:34.995771    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.046844 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.046810    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-ca85db\"/\"kube-root-ca.crt\""
Nov 02 22:20:35.066262 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.066215    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-home\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.066463 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.066264    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7gt9\" (UniqueName: \"kubernetes.io/projected/2e148426-ee8e-477a-afcf-9e1674b4e056-kube-api-access-k7gt9\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.066572 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.066545    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e148426-ee8e-477a-afcf-9e1674b4e056-tls-certs\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.066644 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.066627    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-dshm\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.066777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.066753    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-tmp-dir\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.066842 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.066797    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-model-cache\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.093279 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.093246    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ca85db\"/\"default-dockercfg-5tbpv\""
Nov 02 22:20:35.093519 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.093336    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ca85db\"/\"premium-w1-ca85db-kserve-self-signed-certs\""
Nov 02 22:20:35.093519 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.093405    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-ca85db\"/\"openshift-service-ca.crt\""
Nov 02 22:20:35.095951 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.095926    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"]
Nov 02 22:20:35.167995 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.167952    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-dshm\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.168207 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.168032    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-tmp-dir\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.168207 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.168067    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-model-cache\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.168207 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.168112    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-home\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.168207 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.168139    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7gt9\" (UniqueName: \"kubernetes.io/projected/2e148426-ee8e-477a-afcf-9e1674b4e056-kube-api-access-k7gt9\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.168459 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.168243    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e148426-ee8e-477a-afcf-9e1674b4e056-tls-certs\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.169121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.169093    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-model-cache\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.169724 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.169699    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-tmp-dir\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.170528 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.170476    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-home\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.174924 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.174902    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e148426-ee8e-477a-afcf-9e1674b4e056-tls-certs\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.179712 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.179683    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-dshm\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.228400 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.228333    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7gt9\" (UniqueName: \"kubernetes.io/projected/2e148426-ee8e-477a-afcf-9e1674b4e056-kube-api-access-k7gt9\") pod \"premium-w1-ca85db-kserve-f96fc4cc8-wskqj\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") " pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.308071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.307981    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:35.727523 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:35.727475    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"]
Nov 02 22:20:35.730002 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:20:35.729967    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e148426_ee8e_477a_afcf_9e1674b4e056.slice/crio-d25fbc0ebec596962b941e478f3d59d92451bd728ec9965a093edb85363d11d5 WatchSource:0}: Error finding container d25fbc0ebec596962b941e478f3d59d92451bd728ec9965a093edb85363d11d5: Status 404 returned error can't find the container with id d25fbc0ebec596962b941e478f3d59d92451bd728ec9965a093edb85363d11d5
Nov 02 22:20:36.550326 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:36.550286    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" event={"ID":"2e148426-ee8e-477a-afcf-9e1674b4e056","Type":"ContainerStarted","Data":"5b7840ca1281ba9d9b5e95c1486f2c48ffcc316be5eabb3f5e11eee4f5a91212"}
Nov 02 22:20:36.550326 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:36.550334    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" event={"ID":"2e148426-ee8e-477a-afcf-9e1674b4e056","Type":"ContainerStarted","Data":"d25fbc0ebec596962b941e478f3d59d92451bd728ec9965a093edb85363d11d5"}
Nov 02 22:20:36.665158 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:36.665104    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podStartSLOduration=2.6650886529999998 podStartE2EDuration="2.665088653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:20:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:20:36.663700512 +0000 UTC m=+908.947510199" watchObservedRunningTime="2026-11-02 22:20:36.665088653 +0000 UTC m=+908.948898336"
Nov 02 22:20:37.308146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:37.308102    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:37.579480 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:20:37.579397    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:20:45.308142 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:45.308106    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:45.324610 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:45.324578    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:20:45.627023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:20:45.626947    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:21:07.577704 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:21:07.577664    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:21:37.578931 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:21:37.578888    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:21:38.418630 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.418594    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"]
Nov 02 22:21:38.422963 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.422943    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.471791 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.471761    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-8263a4-openshift-default-dockercfg-t2p57\""
Nov 02 22:21:38.516045 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.515995    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-envoy\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.516244 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.516167    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl4tg\" (UniqueName: \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-kube-api-access-vl4tg\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.516244 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.516214    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-istio-token\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.516415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.516255    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/966ac637-1148-482f-9bb6-d877b0aed346-istiod-ca-cert\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.516415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.516355    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-credential-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.516538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.516481    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/966ac637-1148-482f-9bb6-d877b0aed346-istio-podinfo\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.516538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.516521    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.516648 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.516548    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-data\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.516648 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.516579    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-certs\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.520398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.520374    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"]
Nov 02 22:21:38.617654 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.617615    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl4tg\" (UniqueName: \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-kube-api-access-vl4tg\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.618280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.617666    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-istio-token\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.618280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.617818    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/966ac637-1148-482f-9bb6-d877b0aed346-istiod-ca-cert\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.618280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.617942    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-credential-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.618280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.617975    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/966ac637-1148-482f-9bb6-d877b0aed346-istio-podinfo\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.618280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.618003    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.618280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.618034    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-data\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.618280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.618086    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-certs\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.618280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.618131    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-envoy\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.618745 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.618408    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-credential-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.618745 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.618531    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-data\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.619075 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.619051    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-certs\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.619280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.619256    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.619533 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.619514    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/966ac637-1148-482f-9bb6-d877b0aed346-istiod-ca-cert\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.620576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.620554    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-envoy\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.621020 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.621001    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/966ac637-1148-482f-9bb6-d877b0aed346-istio-podinfo\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.676134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.676056    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl4tg\" (UniqueName: \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-kube-api-access-vl4tg\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.676134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.676098    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-istio-token\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:38.738987 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:38.738941    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:39.172741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:39.172713    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"]
Nov 02 22:21:39.176210 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:21:39.176175    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966ac637_1148_482f_9bb6_d877b0aed346.slice/crio-9319e4c75ab4254586d3b67dc217cb418570897086b1689cec05e3a727901503 WatchSource:0}: Error finding container 9319e4c75ab4254586d3b67dc217cb418570897086b1689cec05e3a727901503: Status 404 returned error can't find the container with id 9319e4c75ab4254586d3b67dc217cb418570897086b1689cec05e3a727901503
Nov 02 22:21:39.179074 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:39.179035    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:21:39.179181 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:39.179124    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:21:39.179181 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:39.179166    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:21:39.973898 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:39.973852    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" event={"ID":"966ac637-1148-482f-9bb6-d877b0aed346","Type":"ContainerStarted","Data":"273903ee1db696727e323e8d4162540e95c4a5b1fd598145124ed6e3e1bc3d68"}
Nov 02 22:21:39.974315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:39.973927    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" event={"ID":"966ac637-1148-482f-9bb6-d877b0aed346","Type":"ContainerStarted","Data":"9319e4c75ab4254586d3b67dc217cb418570897086b1689cec05e3a727901503"}
Nov 02 22:21:40.744769 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:40.744733    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:40.753551 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:40.753513    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 22:21:40.753827 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:40.753797    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 22:21:41.742393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:41.741788    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 22:21:41.742393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:41.741856    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 22:21:42.568073 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.568014    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" podStartSLOduration=4.567998099 podStartE2EDuration="4.567998099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:21:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:21:40.162768469 +0000 UTC m=+972.446578157" watchObservedRunningTime="2026-11-02 22:21:42.567998099 +0000 UTC m=+974.851807789"
Nov 02 22:21:42.569056 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.569037    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"]
Nov 02 22:21:42.573165 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.573136    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.693418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.691968    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.693418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.692021    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqwrg\" (UniqueName: \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-kube-api-access-vqwrg\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.693418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.692073    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.693418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.692119    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.693418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.692178    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.693418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.692208    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.693418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.692263    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.693418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.692297    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.693418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.692391    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.781093 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.779892    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 22:21:42.781093 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.779953    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 22:21:42.781093 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.780572    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"]
Nov 02 22:21:42.793525 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.793498    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.793718 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.793534    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.793718 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.793572    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.793718 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.793595    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.793718 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.793644    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.793718 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.793677    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.793718 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.793698    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqwrg\" (UniqueName: \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-kube-api-access-vqwrg\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.793915 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.793733    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.793915 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.793761    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.794565 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.794050    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.794565 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.794129    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.794565 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.794268    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.794565 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.794347    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.794565 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.794492    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.797331 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.796756    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.797495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.797412    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.861393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.861274    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqwrg\" (UniqueName: \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-kube-api-access-vqwrg\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.861972 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.861949    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-cbgcz\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:42.891928 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:42.891886    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:43.359400 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:43.359313    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"]
Nov 02 22:21:43.360259 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:21:43.360225    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818e9335_6d03_43c4_85a8_191d2ed5c6d1.slice/crio-9460a62758af2a9f4bebd0d354151a3d93270d7d0d932101cadc79bbfb5a308e WatchSource:0}: Error finding container 9460a62758af2a9f4bebd0d354151a3d93270d7d0d932101cadc79bbfb5a308e: Status 404 returned error can't find the container with id 9460a62758af2a9f4bebd0d354151a3d93270d7d0d932101cadc79bbfb5a308e
Nov 02 22:21:43.363192 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:43.363159    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:21:43.363295 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:43.363225    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:21:43.363295 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:43.363254    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:21:43.744390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:43.742897    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 22:21:43.744390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:43.742974    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 22:21:44.002119 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:44.002029    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" event={"ID":"818e9335-6d03-43c4-85a8-191d2ed5c6d1","Type":"ContainerStarted","Data":"0782744162a2f8940060fe87bbf305f3d3e5c9f15f4cf7aef78c7bf7592059cf"}
Nov 02 22:21:44.002119 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:44.002081    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" event={"ID":"818e9335-6d03-43c4-85a8-191d2ed5c6d1","Type":"ContainerStarted","Data":"9460a62758af2a9f4bebd0d354151a3d93270d7d0d932101cadc79bbfb5a308e"}
Nov 02 22:21:44.117815 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:44.117356    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" podStartSLOduration=2.117341928 podStartE2EDuration="2.117341928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:21:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:21:44.115009869 +0000 UTC m=+976.398819587" watchObservedRunningTime="2026-11-02 22:21:44.117341928 +0000 UTC m=+976.401151615"
Nov 02 22:21:44.750390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:44.745638    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 22:21:44.750390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:44.745710    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 22:21:44.894429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:44.894356    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:44.894634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:44.894551    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-cbgcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 22:21:44.894634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:44.894598    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 22:21:45.744391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:45.741695    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 02 22:21:45.744391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:45.741756    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 02 22:21:45.892917 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:45.892873    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-cbgcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 22:21:45.893097 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:45.892931    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 22:21:46.892600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:46.892561    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-cbgcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 22:21:46.893105 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:46.892622    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 22:21:47.289181 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:47.288878    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:47.289588 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:47.289571    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:47.295404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:47.294846    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:21:47.892512 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:47.892479    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-cbgcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 22:21:47.892697 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:47.892530    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 22:21:48.892355 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:48.892316    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-cbgcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 22:21:48.892551 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:48.892403    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 22:21:49.893310 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:49.893275    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-cbgcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 22:21:49.893757 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:49.893343    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 22:21:50.892637 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:50.892600    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-cbgcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 22:21:50.892854 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:50.892655    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 22:21:51.896955 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:51.896923    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:51.897347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:51.897319    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:21:51.898092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:21:51.898068    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:22:07.579047 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:22:07.579001    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:22:08.314276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.314244    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"]
Nov 02 22:22:08.318483 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.318459    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.368059 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.368030    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-711bd4-openshift-default-dockercfg-h26gm\""
Nov 02 22:22:08.417873 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.417832    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"]
Nov 02 22:22:08.450866 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.450828    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-data\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.451052 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.450880    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istiod-ca-cert\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.451052 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.450972    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-credential-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.451180 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.451055    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-927bl\" (UniqueName: \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-kube-api-access-927bl\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.451180 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.451083    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-certs\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.451180 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.451111    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-token\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.451345 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.451246    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.451345 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.451285    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-envoy\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.451486 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.451394    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-podinfo\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.552868 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.552830    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istiod-ca-cert\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553082 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.552880    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-credential-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553082 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.552923    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-927bl\" (UniqueName: \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-kube-api-access-927bl\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553082 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.552951    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-certs\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553082 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.552996    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-token\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553082 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.553051    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553338 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.553088    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-envoy\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553338 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.553126    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-podinfo\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553338 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.553185    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-data\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553555 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.553519    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553742 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.553722    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-data\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553810 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.553749    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istiod-ca-cert\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553864 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.553808    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-credential-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.553864 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.553845    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-certs\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.555479 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.555450    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-envoy\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.555697 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.555682    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-podinfo\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.611766 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.611696    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-927bl\" (UniqueName: \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-kube-api-access-927bl\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.611766 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.611711    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-token\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:08.633509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:08.633194    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:09.060928 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:09.060905    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"]
Nov 02 22:22:09.064172 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:22:09.064147    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e2ea44_ad21_46e7_b229_fe7d95bdc1a7.slice/crio-e637ce5cdc15ce1b51e735288f4cfc038c63a8aebe73f30496a1105db293d18e WatchSource:0}: Error finding container e637ce5cdc15ce1b51e735288f4cfc038c63a8aebe73f30496a1105db293d18e: Status 404 returned error can't find the container with id e637ce5cdc15ce1b51e735288f4cfc038c63a8aebe73f30496a1105db293d18e
Nov 02 22:22:09.066656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:09.066627    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:09.066914 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:09.066689    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:09.066914 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:09.066716    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:09.166168 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:09.165986    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" event={"ID":"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7","Type":"ContainerStarted","Data":"1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798"}
Nov 02 22:22:09.166168 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:09.166067    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" event={"ID":"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7","Type":"ContainerStarted","Data":"e637ce5cdc15ce1b51e735288f4cfc038c63a8aebe73f30496a1105db293d18e"}
Nov 02 22:22:09.282358 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:09.282292    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" podStartSLOduration=1.282275485 podStartE2EDuration="1.282275485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:22:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:22:09.281766928 +0000 UTC m=+1001.565576619" watchObservedRunningTime="2026-11-02 22:22:09.282275485 +0000 UTC m=+1001.566085172"
Nov 02 22:22:09.648390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:09.639452    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:10.644292 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:10.644251    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 22:22:10.644636 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:10.644606    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 22:22:11.637800 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:11.637762    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 22:22:11.638423 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:11.638386    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 22:22:12.643392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:12.637729    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 22:22:12.643392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:12.637804    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 22:22:13.645499 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:13.645459    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 22:22:13.646097 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:13.646060    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 22:22:14.635206 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:14.634757    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 02 22:22:14.635206 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:14.634823    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 02 22:22:16.399235 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:16.399202    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:16.399667 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:16.399493    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:16.400274 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:16.400256    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:22:25.043808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.043767    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"]
Nov 02 22:22:25.047989 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.047969    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.097417 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.097356    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-ca85db-openshift-default-dockercfg-bzw5r\""
Nov 02 22:22:25.131979 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.131943    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-credential-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.132139 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.131986    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-envoy\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.132139 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.132056    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-data\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.132139 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.132092    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istiod-ca-cert\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.132264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.132223    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcn4m\" (UniqueName: \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-kube-api-access-mcn4m\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.132264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.132254    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-certs\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.132377 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.132349    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-token\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.132444 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.132399    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.132505 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.132481    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-podinfo\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.147976 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.147944    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"]
Nov 02 22:22:25.233256 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233217    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istiod-ca-cert\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.233498 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233292    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcn4m\" (UniqueName: \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-kube-api-access-mcn4m\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.233498 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233315    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-certs\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.233498 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233353    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-token\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.233498 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233438    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.233741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233510    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-podinfo\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.233741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233605    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-credential-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.233741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233637    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-envoy\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.233741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233663    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-data\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.233967 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233932    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-certs\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.234029 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.233997    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-data\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.234122 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.234100    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istiod-ca-cert\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.234262 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.234241    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-credential-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.234632 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.234609    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.236657 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.236631    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-envoy\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.236871 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.236855    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-podinfo\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.291590 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.291563    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-token\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.291785 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.291762    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcn4m\" (UniqueName: \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-kube-api-access-mcn4m\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.363206 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.363118    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:25.797305 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.797266    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"]
Nov 02 22:22:25.833116 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:22:25.833083    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0068f3ef_ef4d_47d1_a79b_9fa8ea69461a.slice/crio-847363aaf83d9c598007c1645a7b77b68491be1666feb8040974460c687aa24f WatchSource:0}: Error finding container 847363aaf83d9c598007c1645a7b77b68491be1666feb8040974460c687aa24f: Status 404 returned error can't find the container with id 847363aaf83d9c598007c1645a7b77b68491be1666feb8040974460c687aa24f
Nov 02 22:22:25.836244 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.836207    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:25.836416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.836281    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:25.836416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:25.836309    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:26.269741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:26.269696    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" event={"ID":"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a","Type":"ContainerStarted","Data":"921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056"}
Nov 02 22:22:26.270228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:26.269764    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" event={"ID":"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a","Type":"ContainerStarted","Data":"847363aaf83d9c598007c1645a7b77b68491be1666feb8040974460c687aa24f"}
Nov 02 22:22:26.363897 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:26.363860    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:26.365891 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:26.365857    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 22:22:26.366049 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:26.365916    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 22:22:26.466223 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:26.466164    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" podStartSLOduration=1.46614711 podStartE2EDuration="1.46614711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:22:26.430253268 +0000 UTC m=+1018.714062981" watchObservedRunningTime="2026-11-02 22:22:26.46614711 +0000 UTC m=+1018.749956882"
Nov 02 22:22:27.363497 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:27.363463    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 22:22:27.363929 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:27.363514    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 22:22:28.363851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:28.363802    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 22:22:28.364381 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:28.363887    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 22:22:29.365786 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:29.365748    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 22:22:29.366396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:29.366335    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 22:22:30.363462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:30.363419    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 22:22:30.363693 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:30.363478    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 22:22:31.364507 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:31.364471    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 02 22:22:31.365046 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:31.364542    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 02 22:22:32.610972 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:32.610934    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:32.611623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:32.611600    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:32.612395 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:32.612358    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:22:37.590964 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:22:37.590914    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:22:43.625620 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.625583    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"]
Nov 02 22:22:43.629316 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.629298    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.685248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.685211    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"]
Nov 02 22:22:43.687465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.687435    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-dshm\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.687613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.687534    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-home\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.687613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.687568    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tmp-dir\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.687730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.687648    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-model-cache\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.687792 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.687723    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tls-certs\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.687879 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.687850    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d56ns\" (UniqueName: \"kubernetes.io/projected/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-kube-api-access-d56ns\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.728717 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.728682    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-fe810f\"/\"distinct-w5-fe810f-kserve-self-signed-certs\""
Nov 02 22:22:43.789466 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.789432    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-dshm\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.789666 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.789488    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-home\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.789666 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.789512    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tmp-dir\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.789666 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.789551    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-model-cache\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.789666 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.789566    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tls-certs\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.789666 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.789614    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d56ns\" (UniqueName: \"kubernetes.io/projected/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-kube-api-access-d56ns\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.790226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.790020    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-home\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.790226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.790059    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tmp-dir\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.790349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.790243    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-model-cache\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.791984 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.791960    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-dshm\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.792399 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.792359    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tls-certs\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.847436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.847402    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d56ns\" (UniqueName: \"kubernetes.io/projected/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-kube-api-access-d56ns\") pod \"distinct-w5-fe810f-kserve-85c86cb444-7z8js\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") " pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:43.941632 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:43.941544    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:44.338466 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:44.338440    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"]
Nov 02 22:22:44.340544 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:22:44.340498    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3a2f228_12a8_4e72_b32e_9b7fcb223f0b.slice/crio-2a53769cb5b12fc4fc89cade12e6198e1d581b10fca809bd014455286f759873 WatchSource:0}: Error finding container 2a53769cb5b12fc4fc89cade12e6198e1d581b10fca809bd014455286f759873: Status 404 returned error can't find the container with id 2a53769cb5b12fc4fc89cade12e6198e1d581b10fca809bd014455286f759873
Nov 02 22:22:44.384709 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:44.384682    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" event={"ID":"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b","Type":"ContainerStarted","Data":"2a53769cb5b12fc4fc89cade12e6198e1d581b10fca809bd014455286f759873"}
Nov 02 22:22:45.391336 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:45.391298    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" event={"ID":"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b","Type":"ContainerStarted","Data":"028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695"}
Nov 02 22:22:45.501817 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:45.501757    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podStartSLOduration=2.501742589 podStartE2EDuration="2.501742589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:22:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:22:45.500959296 +0000 UTC m=+1037.784769006" watchObservedRunningTime="2026-11-02 22:22:45.501742589 +0000 UTC m=+1037.785552275"
Nov 02 22:22:45.942469 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:45.942433    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:53.345316 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.345277    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"]
Nov 02 22:22:53.349482 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.349458    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.398518 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.398489    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-49a557-openshift-default-dockercfg-lgv8v\""
Nov 02 22:22:53.399987 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.399963    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-credential-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.400101 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.399995    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-certs\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.400101 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.400080    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-token\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.400205 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.400139    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-envoy\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.400205 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.400164    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-podinfo\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.400297 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.400280    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-data\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.400347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.400310    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ed629b-1c29-4cfc-909c-7b4089c141ac-istiod-ca-cert\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.400490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.400471    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ffnv\" (UniqueName: \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-kube-api-access-4ffnv\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.400558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.400520    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.448825 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.448790    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"]
Nov 02 22:22:53.501691 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.501656    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ffnv\" (UniqueName: \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-kube-api-access-4ffnv\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.501871 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.501703    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.501871 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.501734    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-credential-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.501871 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.501751    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-certs\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.501871 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.501781    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-token\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.501871 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.501825    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-envoy\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.501871 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.501859    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-podinfo\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.502187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.501915    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-data\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.502187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.501940    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ed629b-1c29-4cfc-909c-7b4089c141ac-istiod-ca-cert\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.502298 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.502214    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.502354 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.502308    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-certs\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.502424 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.502356    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-data\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.502511 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.502489    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-credential-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.502660 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.502642    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ed629b-1c29-4cfc-909c-7b4089c141ac-istiod-ca-cert\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.504169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.504147    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-envoy\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.504415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.504397    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-podinfo\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.559127 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.559095    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-token\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.559324 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.559301    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ffnv\" (UniqueName: \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-kube-api-access-4ffnv\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.666748 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.666655    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:53.942757 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.942675    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:53.956232 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:53.956202    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:54.000181 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.000146    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"]
Nov 02 22:22:54.005388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.005345    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.087337 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.087300    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"]
Nov 02 22:22:54.101741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.101715    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"]
Nov 02 22:22:54.104441 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:22:54.104408    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ed629b_1c29_4cfc_909c_7b4089c141ac.slice/crio-f20e485be5057e7bb0336c29e3b01e6d4997fb74a1d260b9a056c69741a54c07 WatchSource:0}: Error finding container f20e485be5057e7bb0336c29e3b01e6d4997fb74a1d260b9a056c69741a54c07: Status 404 returned error can't find the container with id f20e485be5057e7bb0336c29e3b01e6d4997fb74a1d260b9a056c69741a54c07
Nov 02 22:22:54.106917 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.106894    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istiod-ca-cert\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.107026 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.106956    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-data\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.107092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.107058    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-envoy\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.107092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.107055    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:54.107204 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.107092    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-token\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.107204 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.107139    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:54.107204 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.107171    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:54.107204 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.107196    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-credential-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.107458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.107261    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-podinfo\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.107458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.107282    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.107458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.107400    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-certs\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.107458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.107432    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpx2d\" (UniqueName: \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-kube-api-access-vpx2d\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.208535 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.208416    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-podinfo\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.208535 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.208471    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.208870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.208566    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-certs\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.208870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.208593    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx2d\" (UniqueName: \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-kube-api-access-vpx2d\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.208870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.208629    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istiod-ca-cert\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.208870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.208749    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-data\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.209077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.209004    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-envoy\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.209077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.209032    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-token\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.209077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.209050    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-certs\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.209245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.209140    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-credential-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.209303 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.209243    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-data\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.209464 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.209440    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istiod-ca-cert\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.209543 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.209469    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-credential-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.209643 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.209628    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.211313 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.211285    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-envoy\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.211714 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.211695    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-podinfo\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.266187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.266157    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-token\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.266758 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.266738    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpx2d\" (UniqueName: \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-kube-api-access-vpx2d\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.320646 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.320615    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:54.444924 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.444818    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" event={"ID":"59ed629b-1c29-4cfc-909c-7b4089c141ac","Type":"ContainerStarted","Data":"94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06"}
Nov 02 22:22:54.444924 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.444880    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" event={"ID":"59ed629b-1c29-4cfc-909c-7b4089c141ac","Type":"ContainerStarted","Data":"f20e485be5057e7bb0336c29e3b01e6d4997fb74a1d260b9a056c69741a54c07"}
Nov 02 22:22:54.459747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.459669    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:22:54.563723 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.563656    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" podStartSLOduration=1.5636343990000001 podStartE2EDuration="1.563634399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:22:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:22:54.559104089 +0000 UTC m=+1046.842913776" watchObservedRunningTime="2026-11-02 22:22:54.563634399 +0000 UTC m=+1046.847444086"
Nov 02 22:22:54.667238 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.667200    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:22:54.756178 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.756147    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"]
Nov 02 22:22:54.758270 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:22:54.758238    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3600f0db_a16b_4af9_beaf_3c357cf73e9d.slice/crio-57d110290bcc58797cbd5199bf17460c38952bb55cd0e316f8a64a36126b9a16 WatchSource:0}: Error finding container 57d110290bcc58797cbd5199bf17460c38952bb55cd0e316f8a64a36126b9a16: Status 404 returned error can't find the container with id 57d110290bcc58797cbd5199bf17460c38952bb55cd0e316f8a64a36126b9a16
Nov 02 22:22:54.761353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.761320    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:54.761519 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.761428    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:54.761519 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:54.761479    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:22:55.472113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:55.472072    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" event={"ID":"3600f0db-a16b-4af9-beaf-3c357cf73e9d","Type":"ContainerStarted","Data":"eb97790949981f471461ef800b00c44b968fbc89fd344b4f3ceb6010ddfa9593"}
Nov 02 22:22:55.472525 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:55.472140    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" event={"ID":"3600f0db-a16b-4af9-beaf-3c357cf73e9d","Type":"ContainerStarted","Data":"57d110290bcc58797cbd5199bf17460c38952bb55cd0e316f8a64a36126b9a16"}
Nov 02 22:22:55.691422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:55.687522    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 22:22:55.691422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:55.687586    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 22:22:56.321294 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:56.321260    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:22:56.322921 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:56.322895    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 22:22:56.323048 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:56.322952    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 22:22:56.675445 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:56.671729    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 22:22:56.675445 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:56.671788    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 22:22:57.321884 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:57.321846    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 22:22:57.322069 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:57.321900    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 22:22:57.685883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:57.685784    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 22:22:57.685883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:57.685849    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 22:22:58.322123 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.322088    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 22:22:58.322665 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.322504    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 22:22:58.330480 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.330424    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" podStartSLOduration=5.330352571 podStartE2EDuration="5.330352571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:22:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:22:55.599038627 +0000 UTC m=+1047.882848326" watchObservedRunningTime="2026-11-02 22:22:58.330352571 +0000 UTC m=+1050.614162259"
Nov 02 22:22:58.350461 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.350422    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-zcs2n-xttvz"]
Nov 02 22:22:58.364890 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.364859    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.413353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.413315    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-zcs2n-xttvz"]
Nov 02 22:22:58.504396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.504325    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-service-ca-bundle\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.504575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.504401    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-trusted-ca-bundle\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.504575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.504507    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/10e54eb6-50fd-4815-b59c-77c945ea2165-archives-path\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.504575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.504535    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxc7s\" (UniqueName: \"kubernetes.io/projected/10e54eb6-50fd-4815-b59c-77c945ea2165-kube-api-access-fxc7s\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.606456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.605617    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-service-ca-bundle\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.606456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.605656    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-trusted-ca-bundle\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.606456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.605734    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/10e54eb6-50fd-4815-b59c-77c945ea2165-archives-path\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.606456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.605754    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxc7s\" (UniqueName: \"kubernetes.io/projected/10e54eb6-50fd-4815-b59c-77c945ea2165-kube-api-access-fxc7s\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.606801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.606528    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-service-ca-bundle\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.606801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.606731    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-trusted-ca-bundle\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.606909 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.606803    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/10e54eb6-50fd-4815-b59c-77c945ea2165-archives-path\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.671390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.671070    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 22:22:58.671390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.671150    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 22:22:58.706398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.706337    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxc7s\" (UniqueName: \"kubernetes.io/projected/10e54eb6-50fd-4815-b59c-77c945ea2165-kube-api-access-fxc7s\") pod \"periodic-gathering-zcs2n-xttvz\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") " pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:58.713632 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:58.713600    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:22:59.318954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:59.318914    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-zcs2n-xttvz"]
Nov 02 22:22:59.321323 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:59.321288    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 22:22:59.321576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:59.321542    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 22:22:59.321674 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:22:59.321331    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e54eb6_50fd_4815_b59c_77c945ea2165.slice/crio-b498ad202371f72fb62eb8fa356109b61da68fe59db483785b1f28a261f63a51 WatchSource:0}: Error finding container b498ad202371f72fb62eb8fa356109b61da68fe59db483785b1f28a261f63a51: Status 404 returned error can't find the container with id b498ad202371f72fb62eb8fa356109b61da68fe59db483785b1f28a261f63a51
Nov 02 22:22:59.518862 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:59.518822    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-zcs2n-xttvz" event={"ID":"10e54eb6-50fd-4815-b59c-77c945ea2165","Type":"ContainerStarted","Data":"1fb2f09cf476a34dd3f8820db1fc788546b240ec8448816f7d2970f26aa8f31e"}
Nov 02 22:22:59.519074 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:59.518886    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-zcs2n-xttvz" event={"ID":"10e54eb6-50fd-4815-b59c-77c945ea2165","Type":"ContainerStarted","Data":"b498ad202371f72fb62eb8fa356109b61da68fe59db483785b1f28a261f63a51"}
Nov 02 22:22:59.630284 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:59.630178    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-zcs2n-xttvz" podStartSLOduration=1.6301594449999999 podStartE2EDuration="1.630159445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:22:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:22:59.62914524 +0000 UTC m=+1051.912954928" watchObservedRunningTime="2026-11-02 22:22:59.630159445 +0000 UTC m=+1051.913969132"
Nov 02 22:22:59.669715 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:59.667486    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 22:22:59.669715 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:22:59.667549    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 22:23:00.321019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:00.320990    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 22:23:00.321594 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:00.321562    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 22:23:00.671437 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:00.670989    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 02 22:23:00.671437 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:00.671064    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 02 22:23:01.322391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:01.322201    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 02 22:23:01.322391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:01.322264    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 02 22:23:01.745170 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:01.744971    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:23:01.745864 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:01.745839    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:23:01.747520 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:01.747492    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:23:02.325043 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:02.325004    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:23:02.325654 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:02.325390    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:23:02.326018 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:02.325999    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:23:07.579440 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:23:07.579397    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:23:09.384201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.384167    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"]
Nov 02 22:23:09.389666 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.389647    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.422458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.422432    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-token\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.422607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.422472    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-envoy\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.422607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.422500    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-244tp\" (UniqueName: \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-kube-api-access-244tp\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.422727 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.422648    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.422727 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.422692    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-data\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.422831 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.422803    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-certs\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.422889 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.422862    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-credential-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.422943 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.422911    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deca162c-c1be-4048-89f4-1ad60fd1ee87-istiod-ca-cert\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.423032 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.423020    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-podinfo\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.484348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.484307    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-fe810f-openshift-default-dockercfg-lk5vp\""
Nov 02 22:23:09.489595 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.489565    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"]
Nov 02 22:23:09.523963 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.523933    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-envoy\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524133 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.523979    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-244tp\" (UniqueName: \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-kube-api-access-244tp\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524133 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.524019    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524133 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.524040    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-data\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524414 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.524355    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-certs\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.524476    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-credential-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524548 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.524500    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-data\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524548 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.524538    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deca162c-c1be-4048-89f4-1ad60fd1ee87-istiod-ca-cert\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524705 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.524677    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-podinfo\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524908 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.524713    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-token\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524908 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.524771    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-credential-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.524908 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.524878    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.525099 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.525022    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-certs\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.525553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.525531    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deca162c-c1be-4048-89f4-1ad60fd1ee87-istiod-ca-cert\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.527345 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.527322    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-envoy\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.527968 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.527936    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-podinfo\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.582722 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.582689    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-token\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.582904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.582869    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-244tp\" (UniqueName: \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-kube-api-access-244tp\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:09.704134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:09.704051    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:10.132117 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:10.132091    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"]
Nov 02 22:23:10.134076 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:23:10.134043    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeca162c_c1be_4048_89f4_1ad60fd1ee87.slice/crio-0032f7089c3231902690a9eb951c310e862c7296f23c09aebfddcd33e201396e WatchSource:0}: Error finding container 0032f7089c3231902690a9eb951c310e862c7296f23c09aebfddcd33e201396e: Status 404 returned error can't find the container with id 0032f7089c3231902690a9eb951c310e862c7296f23c09aebfddcd33e201396e
Nov 02 22:23:10.136632 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:10.136589    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:23:10.136730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:10.136676    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:23:10.136730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:10.136719    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:23:10.593430 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:10.593394    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" event={"ID":"deca162c-c1be-4048-89f4-1ad60fd1ee87","Type":"ContainerStarted","Data":"14da73a16794fd9cb0a6e1f33bd7e15e69d03d55a040b10e2af573d382662852"}
Nov 02 22:23:10.593876 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:10.593455    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" event={"ID":"deca162c-c1be-4048-89f4-1ad60fd1ee87","Type":"ContainerStarted","Data":"0032f7089c3231902690a9eb951c310e862c7296f23c09aebfddcd33e201396e"}
Nov 02 22:23:10.704785 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:10.704754    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:10.708490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:10.708447    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" podStartSLOduration=1.708434762 podStartE2EDuration="1.708434762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:23:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:23:10.707008229 +0000 UTC m=+1062.990817918" watchObservedRunningTime="2026-11-02 22:23:10.708434762 +0000 UTC m=+1062.992244505"
Nov 02 22:23:11.705942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:11.705909    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 02 22:23:11.706344 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:11.705962    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 02 22:23:12.705334 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:12.705298    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 02 22:23:12.705562 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:12.705359    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 02 22:23:13.705390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:13.704732    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 02 22:23:13.705390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:13.704802    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 02 22:23:14.705353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:14.705314    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 02 22:23:14.705553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:14.705396    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 02 22:23:15.705229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:15.705191    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 02 22:23:15.705450 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:15.705254    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 02 22:23:16.704721 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:16.704684    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Nov 02 22:23:16.705133 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:16.704746    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Nov 02 22:23:17.709435 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:17.709404    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:17.709830 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:17.709699    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:17.710414 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:23:17.710395    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:23:37.577053 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:23:37.577002    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:24:07.583891 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:24:07.583848    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:24:08.562491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.562457    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"]
Nov 02 22:24:08.566748 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.566730    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.632387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.632325    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"]
Nov 02 22:24:08.651072 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.651025    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5mds\" (UniqueName: \"kubernetes.io/projected/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-kube-api-access-l5mds\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.651239 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.651106    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tls-certs\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.651310 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.651229    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-model-cache\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.651392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.651333    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-home\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.651461 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.651404    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tmp-dir\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.651582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.651560    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-dshm\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.664317 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.664297    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-49a557\"/\"distinct-2-w6-49a557-kserve-self-signed-certs\""
Nov 02 22:24:08.753263 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.753223    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-model-cache\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.753520 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.753499    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-home\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.753594 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.753535    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tmp-dir\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.753683 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.753668    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-dshm\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.753757 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.753733    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-model-cache\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.753841 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.753738    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5mds\" (UniqueName: \"kubernetes.io/projected/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-kube-api-access-l5mds\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.753994 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.753893    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tls-certs\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.754447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.754422    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-home\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.754683 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.754664    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tmp-dir\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.756730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.756707    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-dshm\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.756730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.756722    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tls-certs\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.811581 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.811545    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5mds\" (UniqueName: \"kubernetes.io/projected/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-kube-api-access-l5mds\") pod \"distinct-2-w6-49a557-kserve-c986f8757-rtp4h\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") " pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:08.879732 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:08.879614    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:09.320225 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:09.320196    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"]
Nov 02 22:24:09.323406 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:24:09.323354    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0eb1db6_6b93_4b5b_be68_ea87fcead2f0.slice/crio-27ca9edcb7ecc488f1c9d7b060085e2d7a498a6d99fc7c19bd384ebfe772c7ca WatchSource:0}: Error finding container 27ca9edcb7ecc488f1c9d7b060085e2d7a498a6d99fc7c19bd384ebfe772c7ca: Status 404 returned error can't find the container with id 27ca9edcb7ecc488f1c9d7b060085e2d7a498a6d99fc7c19bd384ebfe772c7ca
Nov 02 22:24:10.047744 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:10.047706    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" event={"ID":"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0","Type":"ContainerStarted","Data":"5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54"}
Nov 02 22:24:10.048150 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:10.047759    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" event={"ID":"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0","Type":"ContainerStarted","Data":"27ca9edcb7ecc488f1c9d7b060085e2d7a498a6d99fc7c19bd384ebfe772c7ca"}
Nov 02 22:24:10.159051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:10.158994    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podStartSLOduration=2.158977318 podStartE2EDuration="2.158977318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:24:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:24:10.157218541 +0000 UTC m=+1122.441028260" watchObservedRunningTime="2026-11-02 22:24:10.158977318 +0000 UTC m=+1122.442787005"
Nov 02 22:24:10.879863 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:10.879832    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:18.879977 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:18.879938    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:18.893243 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:18.893203    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:19.121128 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:19.121096    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:24:35.207224 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:35.207185    5067 generic.go:356] "Generic (PLEG): container finished" podID="10e54eb6-50fd-4815-b59c-77c945ea2165" containerID="1fb2f09cf476a34dd3f8820db1fc788546b240ec8448816f7d2970f26aa8f31e" exitCode=0
Nov 02 22:24:35.207606 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:35.207261    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-zcs2n-xttvz" event={"ID":"10e54eb6-50fd-4815-b59c-77c945ea2165","Type":"ContainerDied","Data":"1fb2f09cf476a34dd3f8820db1fc788546b240ec8448816f7d2970f26aa8f31e"}
Nov 02 22:24:36.367205 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.367178    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:24:36.410961 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.410927    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10e54eb6-50fd-4815-b59c-77c945ea2165-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/10e54eb6-50fd-4815-b59c-77c945ea2165-archives-path\") pod \"10e54eb6-50fd-4815-b59c-77c945ea2165\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") "
Nov 02 22:24:36.411123 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.410984    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-service-ca-bundle\") pod \"10e54eb6-50fd-4815-b59c-77c945ea2165\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") "
Nov 02 22:24:36.411164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.411151    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-trusted-ca-bundle\") pod \"10e54eb6-50fd-4815-b59c-77c945ea2165\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") "
Nov 02 22:24:36.411300 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.411278    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10e54eb6-50fd-4815-b59c-77c945ea2165-kube-api-access-fxc7s\" (UniqueName: \"kubernetes.io/projected/10e54eb6-50fd-4815-b59c-77c945ea2165-kube-api-access-fxc7s\") pod \"10e54eb6-50fd-4815-b59c-77c945ea2165\" (UID: \"10e54eb6-50fd-4815-b59c-77c945ea2165\") "
Nov 02 22:24:36.411411 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.411283    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10e54eb6-50fd-4815-b59c-77c945ea2165-archives-path" pod "10e54eb6-50fd-4815-b59c-77c945ea2165" (UID: "10e54eb6-50fd-4815-b59c-77c945ea2165"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:24:36.411484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.411400    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-service-ca-bundle" pod "10e54eb6-50fd-4815-b59c-77c945ea2165" (UID: "10e54eb6-50fd-4815-b59c-77c945ea2165"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:24:36.411578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.411543    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-trusted-ca-bundle" pod "10e54eb6-50fd-4815-b59c-77c945ea2165" (UID: "10e54eb6-50fd-4815-b59c-77c945ea2165"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:24:36.411805 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.411791    5067 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/10e54eb6-50fd-4815-b59c-77c945ea2165-archives-path\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:24:36.411844 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.411809    5067 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-service-ca-bundle\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:24:36.411844 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.411818    5067 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e54eb6-50fd-4815-b59c-77c945ea2165-trusted-ca-bundle\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:24:36.413407 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.413377    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10e54eb6-50fd-4815-b59c-77c945ea2165-kube-api-access-fxc7s" pod "10e54eb6-50fd-4815-b59c-77c945ea2165" (UID: "10e54eb6-50fd-4815-b59c-77c945ea2165"). InnerVolumeSpecName "kube-api-access-fxc7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:24:36.513100 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:36.513044    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxc7s\" (UniqueName: \"kubernetes.io/projected/10e54eb6-50fd-4815-b59c-77c945ea2165-kube-api-access-fxc7s\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:24:37.220844 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:37.220815    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-zcs2n-xttvz"
Nov 02 22:24:37.221035 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:37.220854    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-zcs2n-xttvz" event={"ID":"10e54eb6-50fd-4815-b59c-77c945ea2165","Type":"ContainerDied","Data":"b498ad202371f72fb62eb8fa356109b61da68fe59db483785b1f28a261f63a51"}
Nov 02 22:24:37.221035 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:37.220903    5067 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b498ad202371f72fb62eb8fa356109b61da68fe59db483785b1f28a261f63a51"
Nov 02 22:24:37.578473 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:24:37.578387    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:24:39.149552 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.149516    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"]
Nov 02 22:24:39.150191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.150094    5067 cpu_manager.go:414] "containerMap: removing container" podUID="10e54eb6-50fd-4815-b59c-77c945ea2165" containerName="insights-gathering"
Nov 02 22:24:39.150191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.150105    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10e54eb6-50fd-4815-b59c-77c945ea2165" containerName="insights-gathering"
Nov 02 22:24:39.150315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.150208    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="10e54eb6-50fd-4815-b59c-77c945ea2165" containerName="insights-gathering"
Nov 02 22:24:39.153445 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.153425    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.247283 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.247244    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-data\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.247488 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.247286    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-podinfo\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.247488 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.247327    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istiod-ca-cert\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.247488 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.247429    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-certs\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.247488 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.247472    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-envoy\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.247712 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.247535    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-token\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.247712 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.247596    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g64rv\" (UniqueName: \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-kube-api-access-g64rv\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.247712 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.247686    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-credential-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.247867 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.247720    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.348700 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.348658    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-certs\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.348883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.348720    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-envoy\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.348883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.348779    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-token\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.348883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.348833    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g64rv\" (UniqueName: \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-kube-api-access-g64rv\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.349077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.348910    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-credential-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.349077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.348945    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.349196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.349156    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-certs\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.349196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.349173    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-data\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.349300 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.349216    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-podinfo\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.349300 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.349249    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.349300 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.349262    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istiod-ca-cert\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.349498 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.349477    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-credential-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.349607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.349582    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-data\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.349986 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.349963    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istiod-ca-cert\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.351633 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.351604    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-envoy\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.352017 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.351993    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-podinfo\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.357959 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.357929    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"]
Nov 02 22:24:39.407272 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.407202    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-token\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.407481 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.407456    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g64rv\" (UniqueName: \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-kube-api-access-g64rv\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.470337 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.470297    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:39.901609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.901578    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"]
Nov 02 22:24:39.903584 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:24:39.903558    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode71caf86_9d04_4442_bcfd_5b7bf21900e6.slice/crio-c249770cc1d1c0713b0afec2c113cb3ab9be954d6895097eb9771c6943a4b925 WatchSource:0}: Error finding container c249770cc1d1c0713b0afec2c113cb3ab9be954d6895097eb9771c6943a4b925: Status 404 returned error can't find the container with id c249770cc1d1c0713b0afec2c113cb3ab9be954d6895097eb9771c6943a4b925
Nov 02 22:24:39.906268 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.906222    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:24:39.906397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.906312    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:24:39.906397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:39.906352    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:24:40.244390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:40.243837    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" event={"ID":"e71caf86-9d04-4442-bcfd-5b7bf21900e6","Type":"ContainerStarted","Data":"11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2"}
Nov 02 22:24:40.244390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:40.243917    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" event={"ID":"e71caf86-9d04-4442-bcfd-5b7bf21900e6","Type":"ContainerStarted","Data":"c249770cc1d1c0713b0afec2c113cb3ab9be954d6895097eb9771c6943a4b925"}
Nov 02 22:24:40.363019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:40.362942    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" podStartSLOduration=1.362923847 podStartE2EDuration="1.362923847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:24:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:24:40.35788879 +0000 UTC m=+1152.641698480" watchObservedRunningTime="2026-11-02 22:24:40.362923847 +0000 UTC m=+1152.646733535"
Nov 02 22:24:40.471454 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:40.471417    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:40.473429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:40.473393    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 22:24:40.473608 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:40.473441    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 22:24:41.470982 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:41.470939    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 22:24:41.471446 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:41.471013    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 22:24:42.471390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:42.471096    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 22:24:42.471390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:42.471172    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 22:24:43.018639 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.018598    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"]
Nov 02 22:24:43.022765 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.022748    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.080422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.080385    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-8263a4\"/\"distinct-2-w2-8263a4-kserve-self-signed-certs\""
Nov 02 22:24:43.086093 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.086058    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"]
Nov 02 22:24:43.100832 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.100801    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-model-cache\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.100832 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.100835    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2ctz\" (UniqueName: \"kubernetes.io/projected/7f0b9e55-929d-467a-b407-00ecfda6f7ab-kube-api-access-x2ctz\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.101085 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.100987    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-home\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.101085 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.101039    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-dshm\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.101201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.101102    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tmp-dir\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.101201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.101125    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tls-certs\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.203041 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.203006    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-home\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.203229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.203067    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-dshm\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.203229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.203108    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tmp-dir\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.203229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.203127    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tls-certs\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.203229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.203219    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-model-cache\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.203531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.203252    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2ctz\" (UniqueName: \"kubernetes.io/projected/7f0b9e55-929d-467a-b407-00ecfda6f7ab-kube-api-access-x2ctz\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.203589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.203533    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-home\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.203589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.203547    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tmp-dir\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.203703 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.203631    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-model-cache\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.205752 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.205724    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-dshm\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.205881 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.205831    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tls-certs\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.272610 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.272532    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2ctz\" (UniqueName: \"kubernetes.io/projected/7f0b9e55-929d-467a-b407-00ecfda6f7ab-kube-api-access-x2ctz\") pod \"distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") " pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.352268 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.352233    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:43.474846 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.474805    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 22:24:43.475497 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.475450    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 22:24:43.830062 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:43.830030    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"]
Nov 02 22:24:43.832183 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:24:43.832156    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f0b9e55_929d_467a_b407_00ecfda6f7ab.slice/crio-4d22c6b7a3dcc61617d317813beb47455a5897010b3a8867365491c78b40600e WatchSource:0}: Error finding container 4d22c6b7a3dcc61617d317813beb47455a5897010b3a8867365491c78b40600e: Status 404 returned error can't find the container with id 4d22c6b7a3dcc61617d317813beb47455a5897010b3a8867365491c78b40600e
Nov 02 22:24:44.287691 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:44.287656    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" event={"ID":"7f0b9e55-929d-467a-b407-00ecfda6f7ab","Type":"ContainerStarted","Data":"bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c"}
Nov 02 22:24:44.287878 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:44.287700    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" event={"ID":"7f0b9e55-929d-467a-b407-00ecfda6f7ab","Type":"ContainerStarted","Data":"4d22c6b7a3dcc61617d317813beb47455a5897010b3a8867365491c78b40600e"}
Nov 02 22:24:44.360502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:44.360458    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:44.404228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:44.404169    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podStartSLOduration=2.404153009 podStartE2EDuration="2.404153009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:24:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:24:44.402447666 +0000 UTC m=+1156.686257376" watchObservedRunningTime="2026-11-02 22:24:44.404153009 +0000 UTC m=+1156.687962695"
Nov 02 22:24:44.470629 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:44.470594    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 22:24:44.470812 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:44.470649    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 22:24:45.471153 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:45.471120    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 22:24:45.471575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:45.471172    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 22:24:46.471497 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:46.471457    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 02 22:24:46.471885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:46.471534    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 02 22:24:47.476027 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:47.475990    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:47.476558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:47.476386    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:47.477281 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:47.477259    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:24:53.353412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:53.353345    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:53.366862 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:53.366834    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:24:54.369474 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:24:54.369439    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:25:07.578726 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:25:07.578682    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:25:10.612902 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.612864    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"]
Nov 02 22:25:10.617825 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.617802    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.666860 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.666809    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-3224ea-openshift-default-dockercfg-2whsf\""
Nov 02 22:25:10.712841 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.712802    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-istio-token\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.713028 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.712928    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mrzb\" (UniqueName: \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-kube-api-access-2mrzb\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.713028 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.712974    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.713352 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.713329    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-data\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.713525 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.713391    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-envoy\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.713525 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.713457    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-credential-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.713525 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.713501    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-certs\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.713679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.713526    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17acd4de-db2e-4d59-a748-a78da66499c9-istio-podinfo\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.713679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.713548    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17acd4de-db2e-4d59-a748-a78da66499c9-istiod-ca-cert\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.719399 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.718320    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"]
Nov 02 22:25:10.814578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.814543    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mrzb\" (UniqueName: \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-kube-api-access-2mrzb\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.814578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.814584    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.814852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.814725    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-data\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.814852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.814747    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-envoy\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.814852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.814765    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-credential-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.814852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.814790    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-certs\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.814852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.814811    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17acd4de-db2e-4d59-a748-a78da66499c9-istio-podinfo\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.814852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.814835    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17acd4de-db2e-4d59-a748-a78da66499c9-istiod-ca-cert\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.815172 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.814877    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-istio-token\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.815172 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.815124    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.815172 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.815163    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-credential-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.815338 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.815224    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-certs\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.815448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.815418    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-data\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.815836 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.815816    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17acd4de-db2e-4d59-a748-a78da66499c9-istiod-ca-cert\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.817102 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.817079    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-envoy\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.817332 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.817310    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17acd4de-db2e-4d59-a748-a78da66499c9-istio-podinfo\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.872857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.872782    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-istio-token\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.873496 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.873470    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mrzb\" (UniqueName: \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-kube-api-access-2mrzb\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:10.933651 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:10.933615    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:11.366152 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:11.366120    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"]
Nov 02 22:25:11.368169 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:25:11.368140    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17acd4de_db2e_4d59_a748_a78da66499c9.slice/crio-dd8cb12e3eac46b4399d28e05d1939e212c595f19b3bd78f1d6f4116d23db69e WatchSource:0}: Error finding container dd8cb12e3eac46b4399d28e05d1939e212c595f19b3bd78f1d6f4116d23db69e: Status 404 returned error can't find the container with id dd8cb12e3eac46b4399d28e05d1939e212c595f19b3bd78f1d6f4116d23db69e
Nov 02 22:25:11.370979 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:11.370943    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:11.371116 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:11.371022    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:11.371116 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:11.371094    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:11.488995 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:11.488954    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" event={"ID":"17acd4de-db2e-4d59-a748-a78da66499c9","Type":"ContainerStarted","Data":"ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01"}
Nov 02 22:25:11.489196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:11.489036    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" event={"ID":"17acd4de-db2e-4d59-a748-a78da66499c9","Type":"ContainerStarted","Data":"dd8cb12e3eac46b4399d28e05d1939e212c595f19b3bd78f1d6f4116d23db69e"}
Nov 02 22:25:11.607203 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:11.607142    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" podStartSLOduration=1.6071231369999999 podStartE2EDuration="1.607123137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:25:11.602413766 +0000 UTC m=+1183.886223480" watchObservedRunningTime="2026-11-02 22:25:11.607123137 +0000 UTC m=+1183.890932826"
Nov 02 22:25:11.937407 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:11.934519    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:12.958566 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:12.958519    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 22:25:12.959112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:12.958590    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 22:25:13.956387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:13.955754    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 22:25:13.956387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:13.955832    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 22:25:14.940436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:14.936761    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 22:25:14.940436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:14.936828    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 22:25:15.956391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:15.947764    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 22:25:15.956391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:15.947833    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 22:25:16.938389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:16.937741    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 22:25:16.938389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:16.937816    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 22:25:17.934466 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:17.934429    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 22:25:17.934889 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:17.934483    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 22:25:18.938593 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:18.938559    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:18.938997 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:18.938810    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:18.939602 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:18.939583    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:25:20.913415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:20.913377    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"]
Nov 02 22:25:20.917683 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:20.917661    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:20.969625 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:20.969587    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tls-certs\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:20.969924 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:20.969904    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tmp-dir\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:20.970074 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:20.970050    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-dshm\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:20.970194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:20.970138    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-model-cache\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:20.970194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:20.970158    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9xlq\" (UniqueName: \"kubernetes.io/projected/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-kube-api-access-m9xlq\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:20.970359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:20.970340    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-home\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.015834 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.015797    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-711bd4\"/\"distinct-2-w0-711bd4-kserve-self-signed-certs\""
Nov 02 22:25:21.019601 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.019576    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"]
Nov 02 22:25:21.073561 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.073523    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-dshm\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.073808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.073650    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-model-cache\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.073808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.073746    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9xlq\" (UniqueName: \"kubernetes.io/projected/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-kube-api-access-m9xlq\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.073955 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.073821    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-home\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.074011 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.073959    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tls-certs\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.074064 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.074056    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tmp-dir\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.074497 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.074473    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tmp-dir\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.074814 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.074789    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-home\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.075246 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.075219    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-model-cache\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.076867 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.076845    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-dshm\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.081243 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.081164    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tls-certs\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.133633 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.133598    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9xlq\" (UniqueName: \"kubernetes.io/projected/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-kube-api-access-m9xlq\") pod \"distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") " pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.231100 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.231056    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:21.643107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:21.643082    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"]
Nov 02 22:25:21.645029 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:25:21.644999    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeaad5c27_f022_46cd_9f1a_a8f3390fe0a5.slice/crio-82e30794bc1aaf76feaee70b31dca9d5eebded49523975510cb643be2da705da WatchSource:0}: Error finding container 82e30794bc1aaf76feaee70b31dca9d5eebded49523975510cb643be2da705da: Status 404 returned error can't find the container with id 82e30794bc1aaf76feaee70b31dca9d5eebded49523975510cb643be2da705da
Nov 02 22:25:22.576024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:22.575986    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" event={"ID":"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5","Type":"ContainerStarted","Data":"e86f37a930f7ebe0b74347037fbcf5065f5bfd12c2b9bedabc9f7ddd0109714a"}
Nov 02 22:25:22.576478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:22.576033    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" event={"ID":"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5","Type":"ContainerStarted","Data":"82e30794bc1aaf76feaee70b31dca9d5eebded49523975510cb643be2da705da"}
Nov 02 22:25:22.688038 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:22.687985    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podStartSLOduration=2.68797021 podStartE2EDuration="2.68797021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:25:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:25:22.68591227 +0000 UTC m=+1194.969721982" watchObservedRunningTime="2026-11-02 22:25:22.68797021 +0000 UTC m=+1194.971779897"
Nov 02 22:25:23.232071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:23.232034    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:25.093082 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.093051    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"]
Nov 02 22:25:25.097323 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.097304    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.134029 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.133993    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-token\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.134029 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.134030    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istiod-ca-cert\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.134222 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.134050    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-podinfo\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.134222 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.134193    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-envoy\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.134340 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.134249    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-certs\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.134340 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.134295    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-data\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.134474 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.134453    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.134516 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.134506    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwckr\" (UniqueName: \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-kube-api-access-zwckr\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.134619 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.134601    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-credential-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.236108 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.236030    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-envoy\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.236108 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.236091    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-certs\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.236415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.236133    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-data\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.236415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.236254    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.236415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.236305    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwckr\" (UniqueName: \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-kube-api-access-zwckr\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.236861 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.236772    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-data\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.237441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.237388    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-certs\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.237835 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.237745    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-credential-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.237835 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.237826    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-token\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.238022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.237866    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istiod-ca-cert\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.238022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.237891    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-podinfo\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.239134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.238401    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-credential-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.239134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.239086    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istiod-ca-cert\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.239617 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.239593    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-socket\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.244016 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.243959    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-envoy\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.244171 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.244133    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-podinfo\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.295225 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.295192    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwckr\" (UniqueName: \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-kube-api-access-zwckr\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.298983 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.298954    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-token\") pod \"e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") " pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.300440 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.300411    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"]
Nov 02 22:25:25.411781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.411699    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:25.843246 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:25:25.843212    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ba9c3b_29f1_488e_bdbe_536c7e375bcf.slice/crio-59f3f6e22e2ba8c2f29f2a6ffd51ffa89a753dd18abb212ba629acbecaab7646 WatchSource:0}: Error finding container 59f3f6e22e2ba8c2f29f2a6ffd51ffa89a753dd18abb212ba629acbecaab7646: Status 404 returned error can't find the container with id 59f3f6e22e2ba8c2f29f2a6ffd51ffa89a753dd18abb212ba629acbecaab7646
Nov 02 22:25:25.846108 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.846074    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:25.846207 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.846161    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:25.846207 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.846190    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:25.848609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:25.848587    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"]
Nov 02 22:25:26.603299 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:26.603261    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" event={"ID":"20ba9c3b-29f1-488e-bdbe-536c7e375bcf","Type":"ContainerStarted","Data":"7848cc03291717a09832e994545f1d726d4a576cfdf85caa3c64b9b3ab00e756"}
Nov 02 22:25:26.603818 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:26.603326    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" event={"ID":"20ba9c3b-29f1-488e-bdbe-536c7e375bcf","Type":"ContainerStarted","Data":"59f3f6e22e2ba8c2f29f2a6ffd51ffa89a753dd18abb212ba629acbecaab7646"}
Nov 02 22:25:27.412808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:27.412773    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:27.414407 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:27.414361    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 02 22:25:27.414533 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:27.414428    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 02 22:25:28.011888 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:28.011858    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:25:28.013162 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:28.013141    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:25:28.412519 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:28.412432    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 02 22:25:28.412519 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:28.412494    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 02 22:25:29.413072 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:29.413032    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 02 22:25:29.413508 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:29.413091    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 02 22:25:30.412122 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:30.412088    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 02 22:25:30.412280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:30.412143    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 02 22:25:31.231967 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:31.231923    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:31.281324 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:31.281291    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:31.393994 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:31.393938    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" podStartSLOduration=6.393923319 podStartE2EDuration="6.393923319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:25:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:25:26.776484609 +0000 UTC m=+1199.060294295" watchObservedRunningTime="2026-11-02 22:25:31.393923319 +0000 UTC m=+1203.677733006"
Nov 02 22:25:31.412423 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:31.412393    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 02 22:25:31.412579 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:31.412444    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 02 22:25:31.678843 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:31.678742    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:25:32.412481 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:32.412449    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 02 22:25:32.412867 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:32.412502    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 02 22:25:33.493601 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:33.493562    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:33.494477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:33.494452    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:33.495419 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:33.495395    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:25:37.577218 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:25:37.577175    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:25:54.981429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:54.981378    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"]
Nov 02 22:25:54.985152 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:54.985133    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.085344 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.084804    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"]
Nov 02 22:25:55.091013 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.090674    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-credential-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.091013 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.090857    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-token\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.091013 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.090909    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-certs\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.091280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.091016    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-podinfo\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.091280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.091180    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-data\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.091280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.091231    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk26n\" (UniqueName: \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-kube-api-access-sk26n\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.091452 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.091274    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istiod-ca-cert\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.091452 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.091429    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-envoy\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.091554 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.091474    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.192686 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.192648    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-envoy\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.192898 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.192710    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.192898 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.192770    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-credential-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.192898 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.192862    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-token\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.193087 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.192904    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-certs\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.193087 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.192928    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-podinfo\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.193087 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.193003    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-data\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.193087 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.193028    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk26n\" (UniqueName: \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-kube-api-access-sk26n\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.193087 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.193075    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istiod-ca-cert\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.193439 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.193253    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.193439 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.193261    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-credential-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.193575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.193556    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-data\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.193784 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.193760    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-certs\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.194121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.194098    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istiod-ca-cert\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.195222 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.195200    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-envoy\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.195565 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.195546    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-podinfo\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.251896 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.251827    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-token\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.251896 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.251852    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk26n\" (UniqueName: \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-kube-api-access-sk26n\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.298811 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.298769    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:55.587287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.587203    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"]
Nov 02 22:25:55.592385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.592356    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.654514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.653652    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"]
Nov 02 22:25:55.699870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.699833    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.700034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.699880    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-credential-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.700034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.699955    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-certs\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.700034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.699995    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm625\" (UniqueName: \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-kube-api-access-fm625\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.700034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.700023    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-envoy\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.700175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.700110    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-token\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.700220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.700205    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-data\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.700256 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.700245    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istiod-ca-cert\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.700353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.700341    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-podinfo\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.751854 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.751823    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"]
Nov 02 22:25:55.753714 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:25:55.753687    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709169a6_5fe4_4f6b_a8ad_b1423a4bd939.slice/crio-d8d30317b9d7bf464c884f8cc605d757c1442184658d9d8669f602b5bb2cc68b WatchSource:0}: Error finding container d8d30317b9d7bf464c884f8cc605d757c1442184658d9d8669f602b5bb2cc68b: Status 404 returned error can't find the container with id d8d30317b9d7bf464c884f8cc605d757c1442184658d9d8669f602b5bb2cc68b
Nov 02 22:25:55.756715 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.756683    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:55.756806 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.756743    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:55.756806 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.756769    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:55.801326 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.801298    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-podinfo\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.801463 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.801404    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.801463 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.801452    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-credential-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.801546 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.801513    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-certs\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.801585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.801556    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm625\" (UniqueName: \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-kube-api-access-fm625\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.801638 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.801597    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-envoy\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.801674 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.801657    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-token\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.801766 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.801748    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-data\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.801836 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.801812    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istiod-ca-cert\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.802946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.802416    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.802946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.802463    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-certs\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.802946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.802895    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-data\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.802946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.802922    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-credential-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.803228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.803209    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istiod-ca-cert\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.803986 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.803946    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" event={"ID":"709169a6-5fe4-4f6b-a8ad-b1423a4bd939","Type":"ContainerStarted","Data":"d8d30317b9d7bf464c884f8cc605d757c1442184658d9d8669f602b5bb2cc68b"}
Nov 02 22:25:55.804225 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.804203    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-envoy\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.804650 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.804630    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-podinfo\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.859968 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.859888    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-token\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.860103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.859987    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm625\" (UniqueName: \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-kube-api-access-fm625\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:55.906420 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:55.906358    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:56.362684 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.362655    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"]
Nov 02 22:25:56.364789 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:25:56.364763    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7326e5f6_1330_453b_99a0_1eae9a0c89fc.slice/crio-791967c8f19d6c590def92f77190dc9db033576e4c30773eb8e099e99351aa24 WatchSource:0}: Error finding container 791967c8f19d6c590def92f77190dc9db033576e4c30773eb8e099e99351aa24: Status 404 returned error can't find the container with id 791967c8f19d6c590def92f77190dc9db033576e4c30773eb8e099e99351aa24
Nov 02 22:25:56.367768 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.367737    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:56.367846 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.367798    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:56.367846 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.367825    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:56.606225 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.606194    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"]
Nov 02 22:25:56.610416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.610395    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.676389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.673403    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"]
Nov 02 22:25:56.715194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.715155    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.715488 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.715466    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-certs\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.715981 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.715958    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-envoy\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.716153 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.716137    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4z87\" (UniqueName: \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-kube-api-access-b4z87\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.716321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.716301    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-credential-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.716477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.716462    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e68eb745-1a75-4eaf-9569-54624bd97223-istiod-ca-cert\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.716613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.716600    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e68eb745-1a75-4eaf-9569-54624bd97223-istio-podinfo\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.716722 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.716709    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-data\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.716835 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.716822    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-istio-token\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.813109 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.813063    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" event={"ID":"7326e5f6-1330-453b-99a0-1eae9a0c89fc","Type":"ContainerStarted","Data":"29a6bf44b255c7af1371bddd91f93f3dbae162f323070dc2e0c53c20a6c7f547"}
Nov 02 22:25:56.813278 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.813127    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" event={"ID":"7326e5f6-1330-453b-99a0-1eae9a0c89fc","Type":"ContainerStarted","Data":"791967c8f19d6c590def92f77190dc9db033576e4c30773eb8e099e99351aa24"}
Nov 02 22:25:56.814976 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.814933    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" event={"ID":"709169a6-5fe4-4f6b-a8ad-b1423a4bd939","Type":"ContainerStarted","Data":"8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052"}
Nov 02 22:25:56.817788 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.817766    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.817910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.817796    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-certs\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.817910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.817813    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-envoy\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.817910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.817842    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4z87\" (UniqueName: \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-kube-api-access-b4z87\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.817910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.817898    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-credential-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.818124 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.817934    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e68eb745-1a75-4eaf-9569-54624bd97223-istiod-ca-cert\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.818124 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.817979    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e68eb745-1a75-4eaf-9569-54624bd97223-istio-podinfo\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.818124 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.818012    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-data\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.818124 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.818049    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-istio-token\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.818729 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.818707    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e68eb745-1a75-4eaf-9569-54624bd97223-istiod-ca-cert\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.819935 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.819887    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-data\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.820047 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.819949    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-credential-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.820196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.820171    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-certs\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.820443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.820415    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.821481 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.821458    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e68eb745-1a75-4eaf-9569-54624bd97223-istio-podinfo\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.822601 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.822579    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-envoy\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.876248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.876175    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4z87\" (UniqueName: \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-kube-api-access-b4z87\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.876467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.876405    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-istio-token\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.906748 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.906713    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:25:56.931773 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.931467    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:56.953403 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:56.945410    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" podStartSLOduration=1.945387668 podStartE2EDuration="1.945387668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:25:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:25:56.932967464 +0000 UTC m=+1229.216777170" watchObservedRunningTime="2026-11-02 22:25:56.945387668 +0000 UTC m=+1229.229197355"
Nov 02 22:25:57.080470 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.079884    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" podStartSLOduration=3.079869414 podStartE2EDuration="3.079869414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:25:57.062641042 +0000 UTC m=+1229.346450730" watchObservedRunningTime="2026-11-02 22:25:57.079869414 +0000 UTC m=+1229.363679098"
Nov 02 22:25:57.300452 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.300338    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:25:57.301792 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.301763    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 22:25:57.301956 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.301820    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 22:25:57.388292 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.388243    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"]
Nov 02 22:25:57.389743 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:25:57.389698    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68eb745_1a75_4eaf_9569_54624bd97223.slice/crio-6f7301d182f1518a67f87f651fdd96b409ca2eb6b924b065fe23dada7d2ba114 WatchSource:0}: Error finding container 6f7301d182f1518a67f87f651fdd96b409ca2eb6b924b065fe23dada7d2ba114: Status 404 returned error can't find the container with id 6f7301d182f1518a67f87f651fdd96b409ca2eb6b924b065fe23dada7d2ba114
Nov 02 22:25:57.393422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.392998    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:57.393422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.393075    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:57.393422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.393109    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:25:57.824964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.823083    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" event={"ID":"e68eb745-1a75-4eaf-9569-54624bd97223","Type":"ContainerStarted","Data":"2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b"}
Nov 02 22:25:57.824964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.823163    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" event={"ID":"e68eb745-1a75-4eaf-9569-54624bd97223","Type":"ContainerStarted","Data":"6f7301d182f1518a67f87f651fdd96b409ca2eb6b924b065fe23dada7d2ba114"}
Nov 02 22:25:57.909380 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.909327    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 22:25:57.909580 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.909403    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 22:25:57.932029 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.932003    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:25:57.999846 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:57.999779    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" podStartSLOduration=1.99975796 podStartE2EDuration="1.99975796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:25:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:25:57.947022276 +0000 UTC m=+1230.230831984" watchObservedRunningTime="2026-11-02 22:25:57.99975796 +0000 UTC m=+1230.283567648"
Nov 02 22:25:58.306999 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:58.306965    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 22:25:58.307299 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:58.307252    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 22:25:58.908385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:58.907808    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 22:25:58.908385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:58.907891    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 22:25:58.944524 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:58.944487    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 22:25:58.948457 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:58.948418    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 22:25:59.308388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:59.307791    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 22:25:59.308388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:59.307856    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 22:25:59.913391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:59.909859    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 22:25:59.913391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:59.909921    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 22:25:59.941390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:59.939466    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 22:25:59.941390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:25:59.939528    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 22:26:00.310388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:00.309949    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 22:26:00.310388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:00.310047    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 22:26:00.910638 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:00.910599    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 22:26:00.911040 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:00.911006    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 22:26:00.933263 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:00.933229    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 22:26:00.933860 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:00.933831    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 22:26:01.313261 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:01.313225    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 22:26:01.313561 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:01.313533    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 22:26:01.910388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:01.909720    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 22:26:01.910388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:01.909784    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 22:26:01.938891 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:01.938852    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 22:26:01.960436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:01.955483    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 22:26:02.303131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:02.302681    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 22:26:02.303131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:02.302753    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 22:26:02.907451 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:02.907409    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 22:26:02.907657 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:02.907478    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 22:26:02.931780 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:02.931745    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 22:26:02.931949 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:02.931808    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 22:26:03.300410 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:03.299542    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 22:26:03.300410 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:03.299626    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 22:26:03.913551 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:03.911750    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 22:26:03.913551 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:03.911812    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 22:26:03.932081 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:03.932044    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 22:26:03.932237 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:03.932124    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 22:26:04.305507 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:04.305465    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:26:04.306235 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:04.306209    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:26:04.306934 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:04.306913    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:26:04.911697 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:04.911664    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:26:04.912407 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:04.912382    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:26:04.913331 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:04.913309    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:26:04.932355 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:04.932327    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 02 22:26:04.932477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:04.932410    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 02 22:26:05.936782 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:05.936745    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:26:05.937331 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:05.937228    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:26:05.938010 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:05.937988    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:26:07.593848 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:26:07.593802    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:26:12.642390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:12.641452    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:12.903024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:12.902915    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:26:13.069713 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:13.068488    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:13.316585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:13.316537    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:13.389399 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:13.388709    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:24.926476 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:24.926438    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"]
Nov 02 22:26:24.930526 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:24.930495    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.032165 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.032121    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-credential-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.032403 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.032173    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-certs\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.032403 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.032330    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-podinfo\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.032547 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.032436    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istiod-ca-cert\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.032606 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.032549    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-envoy\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.032664 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.032601    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-data\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.032717 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.032700    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-token\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.032848 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.032825    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgzqz\" (UniqueName: \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-kube-api-access-lgzqz\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.032911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.032868    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.134537 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.134497    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-credential-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.134738 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.134550    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-certs\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.134738 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.134619    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-podinfo\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.134738 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.134684    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istiod-ca-cert\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.135429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.135074    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-credential-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.135429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.135089    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-certs\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.135429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.135272    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-envoy\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.135429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.135343    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-data\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.135429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.135417    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-token\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.135761 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.135521    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgzqz\" (UniqueName: \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-kube-api-access-lgzqz\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.135761 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.135570    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.138684 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.138656    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istiod-ca-cert\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.138973 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.138925    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-data\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.138973 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.138962    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-envoy\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.139280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.139257    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-socket\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.147004 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.146977    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"]
Nov 02 22:26:25.148203 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.148178    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-podinfo\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.193752 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.193681    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-token\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.193752 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.193743    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgzqz\" (UniqueName: \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-kube-api-access-lgzqz\") pod \"e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") " pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.243841 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.243811    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:25.674625 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.674592    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"]
Nov 02 22:26:25.675783 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:26:25.675756    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a1952c2_04f5_40eb_96a0_93a7e9ea1ff6.slice/crio-5532528ca92e02c05ead91e66f38d5d96e65c0943a08d6fbb18742bc17601288 WatchSource:0}: Error finding container 5532528ca92e02c05ead91e66f38d5d96e65c0943a08d6fbb18742bc17601288: Status 404 returned error can't find the container with id 5532528ca92e02c05ead91e66f38d5d96e65c0943a08d6fbb18742bc17601288
Nov 02 22:26:25.678357 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.678320    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:26:25.678512 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.678418    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:26:25.678512 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:25.678471    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:26:26.197179 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.197091    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"]
Nov 02 22:26:26.207471 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.204619    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.263075 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.263022    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" event={"ID":"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6","Type":"ContainerStarted","Data":"d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa"}
Nov 02 22:26:26.263265 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.263092    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" event={"ID":"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6","Type":"ContainerStarted","Data":"5532528ca92e02c05ead91e66f38d5d96e65c0943a08d6fbb18742bc17601288"}
Nov 02 22:26:26.270022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.269973    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"]
Nov 02 22:26:26.364112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.358241    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slscm\" (UniqueName: \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-kube-api-access-slscm\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.364112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.362650    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-podinfo\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.364112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.362742    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-token\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.364112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.362771    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/017257d4-2273-4bd7-a759-f7537dfda7c5-istiod-ca-cert\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.364112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.362820    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.364112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.362844    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-data\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.364112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.362932    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-certs\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.364112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.362954    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-envoy\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.364112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.363098    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-credential-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.457443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.457238    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" podStartSLOduration=2.457218754 podStartE2EDuration="2.457218754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:26:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:26:26.454687474 +0000 UTC m=+1258.738497158" watchObservedRunningTime="2026-11-02 22:26:26.457218754 +0000 UTC m=+1258.741028441"
Nov 02 22:26:26.465288 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465249    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465294    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-data\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465387    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-certs\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465415    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-envoy\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465510    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-credential-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465746 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465566    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slscm\" (UniqueName: \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-kube-api-access-slscm\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465746 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465733    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-podinfo\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465782    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-token\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465789    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465809    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/017257d4-2273-4bd7-a759-f7537dfda7c5-istiod-ca-cert\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465817    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-data\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.465857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.465833    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-certs\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.466398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.466159    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-credential-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.466725 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.466703    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/017257d4-2273-4bd7-a759-f7537dfda7c5-istiod-ca-cert\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.468602 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.468579    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-podinfo\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.468701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.468683    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-envoy\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.524145 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.524104    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-token\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.524322 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.524221    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slscm\" (UniqueName: \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-kube-api-access-slscm\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.530333 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.530305    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:26.992789 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.992758    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"]
Nov 02 22:26:26.994857 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:26:26.994828    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017257d4_2273_4bd7_a759_f7537dfda7c5.slice/crio-799243be8f9cee76eb92c6e83aecb2478bc3ce4103811a7a184db235190bae0f WatchSource:0}: Error finding container 799243be8f9cee76eb92c6e83aecb2478bc3ce4103811a7a184db235190bae0f: Status 404 returned error can't find the container with id 799243be8f9cee76eb92c6e83aecb2478bc3ce4103811a7a184db235190bae0f
Nov 02 22:26:26.997524 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.997488    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:26:26.997600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.997561    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:26:26.997600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:26.997590    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:26:27.250451 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:27.250119    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:27.250451 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:27.250235    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 02 22:26:27.250451 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:27.250292    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 02 22:26:27.272978 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:27.272939    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" event={"ID":"017257d4-2273-4bd7-a759-f7537dfda7c5","Type":"ContainerStarted","Data":"7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851"}
Nov 02 22:26:27.273169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:27.273004    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" event={"ID":"017257d4-2273-4bd7-a759-f7537dfda7c5","Type":"ContainerStarted","Data":"799243be8f9cee76eb92c6e83aecb2478bc3ce4103811a7a184db235190bae0f"}
Nov 02 22:26:27.388915 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:27.388862    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" podStartSLOduration=1.388847864 podStartE2EDuration="1.388847864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:26:27.386770253 +0000 UTC m=+1259.670579961" watchObservedRunningTime="2026-11-02 22:26:27.388847864 +0000 UTC m=+1259.672657552"
Nov 02 22:26:27.530871 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:27.530792    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:28.247392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:28.245770    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 02 22:26:28.247392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:28.245846    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 02 22:26:28.532354 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:28.532266    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 02 22:26:28.532354 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:28.532316    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 02 22:26:29.258393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:29.250766    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 02 22:26:29.258393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:29.250834    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 02 22:26:29.531297 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:29.531199    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 02 22:26:29.531297 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:29.531262    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 02 22:26:30.258520 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:30.258475    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 02 22:26:30.259143 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:30.259109    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 02 22:26:30.531397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:30.531295    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 02 22:26:30.531397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:30.531344    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 02 22:26:31.249390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:31.246693    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 02 22:26:31.249390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:31.246752    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 02 22:26:31.531076 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:31.530986    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 02 22:26:31.531076 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:31.531038    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 02 22:26:32.256388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:32.246129    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 02 22:26:32.256388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:32.246194    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 02 22:26:32.531738 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:32.531649    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 02 22:26:32.531738 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:32.531705    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 02 22:26:33.481488 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:33.481458    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:33.482887 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:33.482700    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:33.483675 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:33.483653    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:26:33.531035 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:33.531000    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 02 22:26:33.531197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:33.531061    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 02 22:26:34.837050 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:34.837006    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:34.843404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:34.838678    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:34.843404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:34.839539    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:26:37.579234 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:26:37.579190    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:26:45.381387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:45.381313    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:45.488977 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:45.488928    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:45.530479 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:45.530420    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:26:45.631167 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:45.631125    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:45.651056 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:45.650957    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:26:45.696392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:45.695467    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:45.724509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:45.724461    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:45.832390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:45.832121    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:45.915296 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:45.913569    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:26:46.064579 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.064511    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:46.112787 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.112737    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:46.245828 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.245779    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:46.311553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.311491    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:46.389873 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.389824    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:46.392224 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.391038    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:26:46.571028 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.570917    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:46.600192 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.600160    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:26:46.717312 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.602996    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:26:47.082392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.699512    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:26:47.099939 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.099884    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:47.115604 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.115151    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:26:47.115604 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.115256    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:26:47.115604 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.115320    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:26:47.118521 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.704643    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:47.134540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.134506    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:26:47.152390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.699586    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:47.164441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.869339    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:26:47.199240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.871500    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 02 22:26:47.199240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:46.994080    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:26:47.199240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.184753    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:26:47.199240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.184912    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:26:47.199240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.185099    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:26:47.199240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.185186    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:26:47.277392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.273453    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:26:47.277392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.275609    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:26:47.360441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.353772    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:26:47.378388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.363755    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:26:47.431389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.431281    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:26:47.655457 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.645834    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:47.655457 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.655109    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:26:47.707390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.706594    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:47.966450 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.933473    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:26:47.966450 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:47.933556    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:48.110393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.088336    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:26:48.125386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.120319    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:26:48.150392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.128454    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:26:48.150392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.147452    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:48.195393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.168495    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:48.195393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.189008    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 02 22:26:48.195393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.189412    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:26:48.350458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.287621    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:48.420009 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.363393    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:48.428480 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.428450    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:26:48.430801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.430778    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:26:48.461540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.461500    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:26:48.471139 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.471093    5067 patch_prober.go:29] interesting pod/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:26:48.471330 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.471163    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:48.501855 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.501823    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:26:48.694280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.674079    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:26:48.694280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.690455    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:26:48.694280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.690524    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:48.794661 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.794623    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:26:48.795022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:48.794980    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:49.160962 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.160902    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:26:49.174189 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.174140    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:26:49.175768 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.175718    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:26:49.190983 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.190941    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:49.193011 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.192967    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:49.293485 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.293430    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:49.462311 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.462222    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:26:49.464431 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.464395    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:49.590575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.585463    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:26:49.590575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.585532    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:49.704391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.697414    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:26:49.762097 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.743496    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:49.977390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.926347    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": context deadline exceeded"
Nov 02 22:26:49.985971 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:49.985925    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": context deadline exceeded"
Nov 02 22:26:50.062866 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.062500    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:26:50.062866 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.062590    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:50.179390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.170511    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:26:50.184740 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.184686    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:26:50.203227 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.203184    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:50.207424 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.207385    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:50.217601 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.217571    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:26:50.294392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.293910    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:50.463228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.463128    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:26:50.465434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.465399    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:50.528999 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.465851    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:50.578386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.547792    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:26:50.586798 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.586768    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:26:50.612727 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.611581    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:26:50.638109 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.635882    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:26:50.640138 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.640113    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:26:50.645239 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.645212    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:26:50.675313 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:26:50.675276    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:27:07.584066 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:27:07.584009    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:27:11.162339 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.162303    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"]
Nov 02 22:27:11.167784 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.167759    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.260089 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.260049    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"]
Nov 02 22:27:11.275683 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.275645    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06409ed1-ce32-4d21-b844-b18b32615f3d-istiod-ca-cert\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.275855 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.275720    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-token\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.275855 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.275770    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5dq9\" (UniqueName: \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-kube-api-access-p5dq9\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.275855 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.275836    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.276039 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.275882    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-podinfo\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.276039 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.275915    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-data\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.276146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.276065    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-credential-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.276146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.276105    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-certs\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.276259 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.276217    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-envoy\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.377753 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.377710    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06409ed1-ce32-4d21-b844-b18b32615f3d-istiod-ca-cert\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.377958 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.377764    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-token\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.377958 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.377876    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5dq9\" (UniqueName: \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-kube-api-access-p5dq9\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.377958 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.377924    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.378129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.377992    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-podinfo\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.378129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.378057    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-data\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.378129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.378125    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-credential-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.378285 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.378154    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-certs\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.378285 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.378239    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-envoy\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.378422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.378331    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.378485 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.378453    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-data\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.378618 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.378524    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-credential-socket\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.378618 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.378596    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06409ed1-ce32-4d21-b844-b18b32615f3d-istiod-ca-cert\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.378740 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.378601    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-certs\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.380886 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.380858    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-envoy\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.381019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.380912    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-podinfo\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.436058 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.435977    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5dq9\" (UniqueName: \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-kube-api-access-p5dq9\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.436217 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.436065    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-token\") pod \"e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") " pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.485116 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.485082    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:11.528987 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.528953    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"]
Nov 02 22:27:11.533519 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.533490    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.583910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.582220    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-credential-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.583910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.582281    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-token\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.583910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.582431    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istiod-ca-cert\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.583910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.582482    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-certs\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.583910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.582513    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9fp7\" (UniqueName: \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-kube-api-access-m9fp7\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.583910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.582593    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-podinfo\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.583910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.582653    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-data\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.583910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.582749    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.583910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.582798    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-envoy\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.598066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.597976    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"]
Nov 02 22:27:11.684128 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.684082    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.684294 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.684172    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-envoy\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.684594 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.684564    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.684721 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.684683    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-credential-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.684721 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.684712    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-token\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.684814 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.684797    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istiod-ca-cert\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.684857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.684848    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-certs\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.684902 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.684876    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fp7\" (UniqueName: \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-kube-api-access-m9fp7\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.684969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.684955    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-podinfo\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.685006 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.684993    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-credential-socket\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.685053 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.685021    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-data\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.685452 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.685432    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-data\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.685507 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.685491    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-certs\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.685568 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.685548    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istiod-ca-cert\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.686646 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.686602    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-envoy\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.687420 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.687402    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-podinfo\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.742695 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.742662    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9fp7\" (UniqueName: \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-kube-api-access-m9fp7\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.742695 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.742681    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-token\") pod \"e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") " pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.887309 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.887272    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:11.928277 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.928243    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"]
Nov 02 22:27:11.939441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.937411    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"]
Nov 02 22:27:11.939441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.937524    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:11.954350 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.950418    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:11.954350 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.950506    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:11.954350 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.950547    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:11.956531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.956454    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"]
Nov 02 22:27:11.963672 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.963136    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:11.988967 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.988941    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-certs\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:11.989060 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.988996    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-credential-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:11.989060 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.989014    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-data\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:11.989060 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.989050    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkn5l\" (UniqueName: \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-kube-api-access-hkn5l\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:11.992656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.989129    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-token\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:11.992656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.989177    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-envoy\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:11.992656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.989232    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d7463af-b57c-45a2-a286-b7a70fadb525-istiod-ca-cert\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:11.992656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.989260    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-podinfo\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:11.992656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:11.989301    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.001896 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.001866    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"]
Nov 02 22:27:12.020576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.020540    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"]
Nov 02 22:27:12.090629 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.090593    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.090772 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.090647    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istiod-ca-cert\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.090772 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.090690    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-token\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.090901 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.090774    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.090901 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.090809    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-data\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.090901 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.090883    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-certs\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.091065 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.090953    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-certs\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.091065 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.090987    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-credential-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.091065 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.091014    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-data\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.091219 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.091070    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkn5l\" (UniqueName: \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-kube-api-access-hkn5l\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.091219 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.091164    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-envoy\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.091315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.091226    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbz7b\" (UniqueName: \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-kube-api-access-nbz7b\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.092334 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.092307    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-token\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.092498 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.092409    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-envoy\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.092498 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.092459    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d7463af-b57c-45a2-a286-b7a70fadb525-istiod-ca-cert\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.092498 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.092484    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-podinfo\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.092773 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.092510    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-credential-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.092773 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.092540    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-podinfo\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.093649 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.093590    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-credential-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.093649 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.093601    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.093897 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.093881    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-certs\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.094661 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.094543    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-data\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.094993 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.094978    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d7463af-b57c-45a2-a286-b7a70fadb525-istiod-ca-cert\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.099822 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.099755    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-podinfo\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.100325 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.100301    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-envoy\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.162417 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.161580    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkn5l\" (UniqueName: \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-kube-api-access-hkn5l\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.180735 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.178987    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-token\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.187852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.187815    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"]
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.193910    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-credential-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.193965    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-podinfo\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.194011    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istiod-ca-cert\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.194049    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-token\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.194126    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.194155    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-data\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.194261    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-certs\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.194418    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-envoy\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.194476    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbz7b\" (UniqueName: \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-kube-api-access-nbz7b\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.195168    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.195438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.195347    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-credential-socket\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.196628 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.196515    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-data\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.199602 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.199550    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istiod-ca-cert\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.200270 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.200121    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-certs\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.202802 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.202782    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.211715 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.209747    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-envoy\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.211715 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.211492    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-podinfo\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.240393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.239446    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"]
Nov 02 22:27:12.258536 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.258501    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-token\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.260397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.260141    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:12.261418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.261388    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbz7b\" (UniqueName: \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-kube-api-access-nbz7b\") pod \"e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") " pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.293681 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:27:12.293640    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ac378c_d84e_4bc1_9fd6_595af7ad3f97.slice/crio-4f4f2b6468bf1e68723938eef62263d33e432e07ba91b12d4dd90ff7fade4295 WatchSource:0}: Error finding container 4f4f2b6468bf1e68723938eef62263d33e432e07ba91b12d4dd90ff7fade4295: Status 404 returned error can't find the container with id 4f4f2b6468bf1e68723938eef62263d33e432e07ba91b12d4dd90ff7fade4295
Nov 02 22:27:12.294639 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.294589    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"]
Nov 02 22:27:12.295521 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.295496    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-token\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.295642 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.295534    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-certs\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.295642 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.295619    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvdv8\" (UniqueName: \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-kube-api-access-nvdv8\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.295751 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.295673    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-podinfo\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.295751 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.295714    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-data\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.295923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.295752    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b203dd1-35c7-42d9-993d-d515b473a2f1-istiod-ca-cert\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.295923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.295778    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.295923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.295873    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-envoy\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.295923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.295907    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-credential-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.297099 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.297059    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:12.299488 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.299427    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.299692 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.299544    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.299692 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.299594    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.397868 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.397830    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-token\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.398021 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.397883    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-certs\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.398021 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.397981    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvdv8\" (UniqueName: \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-kube-api-access-nvdv8\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.398138 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.398043    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-podinfo\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.398138 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.398089    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-data\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.398138 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.398131    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b203dd1-35c7-42d9-993d-d515b473a2f1-istiod-ca-cert\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.398285 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.398162    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.398285 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.398272    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-envoy\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.398411 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.398307    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-credential-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.398577 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.398554    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-certs\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.400001 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.399978    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b203dd1-35c7-42d9-993d-d515b473a2f1-istiod-ca-cert\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.401140 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.401117    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-credential-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.403689 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.403664    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-socket\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.407765 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.407737    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-data\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.411621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.411599    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-envoy\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.417784 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.413167    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" event={"ID":"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97","Type":"ContainerStarted","Data":"4f4f2b6468bf1e68723938eef62263d33e432e07ba91b12d4dd90ff7fade4295"}
Nov 02 22:27:12.417784 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.417387    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" event={"ID":"06409ed1-ce32-4d21-b844-b18b32615f3d","Type":"ContainerStarted","Data":"3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52"}
Nov 02 22:27:12.417784 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.417440    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" event={"ID":"06409ed1-ce32-4d21-b844-b18b32615f3d","Type":"ContainerStarted","Data":"457202471b23525f00ace1b324c3967d34c35f95f88ee4cfd72db2ee61ed1e2d"}
Nov 02 22:27:12.418889 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.418847    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-podinfo\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.474869 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.474471    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvdv8\" (UniqueName: \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-kube-api-access-nvdv8\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.485815 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.485774    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:12.486237 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.486173    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-token\") pod \"e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") " pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.525840 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.525639    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:12.542582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.541141    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podStartSLOduration=1.5411203439999999 podStartE2EDuration="1.541120344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:27:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:27:12.535618636 +0000 UTC m=+1304.819428326" watchObservedRunningTime="2026-11-02 22:27:12.541120344 +0000 UTC m=+1304.824930032"
Nov 02 22:27:12.699042 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.699008    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"]
Nov 02 22:27:12.704858 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.704819    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.704977 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.704910    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.704977 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.704951    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.768379 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.768340    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"]
Nov 02 22:27:12.769516 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:27:12.769483    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69e0edee_4e5a_4d8a_8c1b_8407f2a8ed04.slice/crio-a18369ba026b780a2193baad454bc90cf00e28d629dad39a8e4ed5f5ec544485 WatchSource:0}: Error finding container a18369ba026b780a2193baad454bc90cf00e28d629dad39a8e4ed5f5ec544485: Status 404 returned error can't find the container with id a18369ba026b780a2193baad454bc90cf00e28d629dad39a8e4ed5f5ec544485
Nov 02 22:27:12.774189 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.774149    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.774284 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.774236    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.774329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.774305    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.963977 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.963876    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"]
Nov 02 22:27:12.983385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.980649    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.983385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.980770    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:12.983385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:12.980813    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:13.489819 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.489779    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" event={"ID":"0b203dd1-35c7-42d9-993d-d515b473a2f1","Type":"ContainerStarted","Data":"35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f"}
Nov 02 22:27:13.490479 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.490439    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" event={"ID":"0b203dd1-35c7-42d9-993d-d515b473a2f1","Type":"ContainerStarted","Data":"ab35b412dd2e78f46d2a593f4a06d30c2db32b85b855ed33684cc3e0dda75475"}
Nov 02 22:27:13.494555 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.494528    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" event={"ID":"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04","Type":"ContainerStarted","Data":"f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397"}
Nov 02 22:27:13.494778 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.494758    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" event={"ID":"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04","Type":"ContainerStarted","Data":"a18369ba026b780a2193baad454bc90cf00e28d629dad39a8e4ed5f5ec544485"}
Nov 02 22:27:13.497155 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.496916    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:13.497155 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.496964    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:13.498230 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.498206    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" event={"ID":"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97","Type":"ContainerStarted","Data":"077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e"}
Nov 02 22:27:13.515611 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.515577    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" event={"ID":"5d7463af-b57c-45a2-a286-b7a70fadb525","Type":"ContainerStarted","Data":"e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b"}
Nov 02 22:27:13.515883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.515866    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" event={"ID":"5d7463af-b57c-45a2-a286-b7a70fadb525","Type":"ContainerStarted","Data":"a8c1d7c85ab559c01b32da4d3947d8578ab0ab7e2ba46b1650b1f4063e178d96"}
Nov 02 22:27:13.527397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.526652    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:13.620402 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.620317    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podStartSLOduration=1.620297698 podStartE2EDuration="1.620297698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:27:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:27:13.616678887 +0000 UTC m=+1305.900488565" watchObservedRunningTime="2026-11-02 22:27:13.620297698 +0000 UTC m=+1305.904107385"
Nov 02 22:27:13.816448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:13.774188    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podStartSLOduration=2.774163603 podStartE2EDuration="2.774163603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:27:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:27:13.731912724 +0000 UTC m=+1306.015722442" watchObservedRunningTime="2026-11-02 22:27:13.774163603 +0000 UTC m=+1306.057973290"
Nov 02 22:27:14.022392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.008666    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:14.022392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.008752    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:14.071458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.047119    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:14.097657 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.097571    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podStartSLOduration=3.097546393 podStartE2EDuration="3.097546393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:27:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:27:13.922467922 +0000 UTC m=+1306.206277610" watchObservedRunningTime="2026-11-02 22:27:14.097546393 +0000 UTC m=+1306.381356080"
Nov 02 22:27:14.274387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.273618    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:14.295604 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.295561    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:14.295935 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.295899    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:14.297611 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.297556    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podStartSLOduration=3.2975155369999998 podStartE2EDuration="3.297515537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:27:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:27:14.066101051 +0000 UTC m=+1306.349910749" watchObservedRunningTime="2026-11-02 22:27:14.297515537 +0000 UTC m=+1306.581325225"
Nov 02 22:27:14.298770 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.298746    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:14.344456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.342507    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:14.344456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.342599    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:14.553391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.551413    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:14.553391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.551489    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:14.570855 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.570651    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:14.570855 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.570727    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:14.671571 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.671458    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:14.680885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.680473    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:14.890507 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.888829    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:14.890507 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.888892    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:14.962624 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:14.962462    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:15.138595 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.133471    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:15.138595 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.133886    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:27:15.167303 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.166474    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:15.229448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.216485    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"]
Nov 02 22:27:15.229768 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.220533    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:27:15.230465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.230431    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy" containerID="cri-o://0782744162a2f8940060fe87bbf305f3d3e5c9f15f4cf7aef78c7bf7592059cf" gracePeriod=30
Nov 02 22:27:15.230744 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.220573    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 02 22:27:15.586446 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.542163    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:27:15.674418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.640553    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:27:15.674418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.641031    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:15.674418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.641086    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:15.674418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.642090    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:15.674418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.642129    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:15.674418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.644355    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:15.674418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.645859    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:15.674418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.646567    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:15.674418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.654851    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:15.674418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.654891    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:15.689308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.689266    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:15.713274 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.700678    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:27:15.713274 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.701104    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:27:15.713274 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.712216    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:15.713274 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.713030    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:27:15.746840 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.713957    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:15.984832 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.983751    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:15.984832 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:15.983817    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:16.256475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.231597    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:16.256475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.246496    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:16.256475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.246947    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:27:16.256475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.249224    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:16.268486 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.261954    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:16.345390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.344528    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:16.345390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.344604    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:16.401647 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.387539    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:16.401647 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.387608    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:16.508445 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.507759    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:16.508445 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.507841    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:16.704231 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.704193    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:27:16.732881 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.732844    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:27:16.733181 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.733164    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:27:16.733325 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.733314    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:27:16.733470 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.733456    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:27:16.735509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.735478    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:27:16.798193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.736660    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:27:16.806392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.803803    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded"
Nov 02 22:27:16.806392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.737167    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:16.818434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.806740    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:27:16.818434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.806812    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:27:16.818434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.737215    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:27:16.818434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.737192    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:27:16.826396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.737280    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:16.826396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.824437    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:16.826396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.736704    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:27:16.826396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.826203    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:27:16.826396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.826265    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:27:16.884453 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.842766    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:16.884453 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.843924    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:27:16.920391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.917482    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:16.920391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:16.917537    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:17.331043 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.330992    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:17.332617 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.332458    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:17.334820 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.334787    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:17.334942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.334849    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:17.335019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.334943    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:17.369396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.367739    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:17.369396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.367805    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:17.369396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.369298    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:27:17.436423 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.432583    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:27:17.587713 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.587608    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:17.588038 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.587993    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:17.721391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.696478    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:17.753600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.753533    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:17.792635 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.792577    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:27:18.092442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.804968    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:18.093104 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.876637    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:18.134439 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.876679    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": context deadline exceeded"
Nov 02 22:27:18.134708 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.876701    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:27:18.135210 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.876736    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:18.188397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.876760    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:27:18.188998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:17.954278    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:27:18.189513 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.058715    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:18.189692 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.189660    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:18.330575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.330539    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:18.330879 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.330833    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:18.368553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.366575    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:18.368553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.366987    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:18.368553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.368116    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:18.555393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.530428    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:18.581391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.556358    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:18.581722 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.581662    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:18.621524 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.599526    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:27:18.642589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.626047    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:18.642589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.627426    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:18.642589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.627566    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:18.642589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.628421    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:18.824399 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.816175    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:27:18.937515 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.937411    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:18.937748 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:18.937720    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:19.221563 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.221523    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:27:19.428763 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.281397    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:19.429300 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.277519    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:27:19.449161 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.281478    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:19.449683 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.281504    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:27:19.450063 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.281440    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:19.450452 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.298215    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:19.468451 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.450588    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:19.533491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.416774    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 02 22:27:19.533491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.416794    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:19.533491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.416742    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:19.533491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.358597    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:19.533491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.528828    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:19.533491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.499766    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:19.533491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.528882    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:19.533491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.527269    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:19.533491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.528918    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:19.571994 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.571952    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:19.627938 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.627893    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:19.880953 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.880856    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:19.895210 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.892740    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:19.907760 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.907716    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:19.936512 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.936460    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:19.953389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:19.939146    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": context deadline exceeded"
Nov 02 22:27:20.283155 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.283116    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:20.283555 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.283520    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:20.312679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.312641    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:20.312956 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.312926    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:20.434533 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.434481    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:20.470389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.434749    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:27:20.471029 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.449732    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:20.485424 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.449770    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:20.493391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.459259    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:20.522393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.459294    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:27:20.523015 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.489118    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:20.523257 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.523226    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:20.548443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.545335    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:20.548443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.545453    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:20.548443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.545579    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:20.548443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.545930    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": context deadline exceeded"
Nov 02 22:27:20.548443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.546005    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 02 22:27:20.548443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.546310    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:20.606335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.606278    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:20.672673 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.669631    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:27:20.757393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.748664    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:27:20.903448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.895910    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:20.958512 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.958465    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:20.963391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:20.959593    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:21.091429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.070478    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:21.305391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.286975    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:21.305391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.287034    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:21.377389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.366037    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:21.377389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.366097    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:21.463412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.451960    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:21.539393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.473812    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:27:21.539393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.479108    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:27:21.539393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.488480    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:21.539393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.491736    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:21.539393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.491787    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:21.539393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.498687    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:21.539393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.525378    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:21.539393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.531913    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:21.539393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.531971    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:21.574448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.563973    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:21.574448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.565153    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:21.643492 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.609500    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:27:21.713497 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.713446    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:27:21.815854 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.815729    5067 generic.go:356] "Generic (PLEG): container finished" podID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerID="0782744162a2f8940060fe87bbf305f3d3e5c9f15f4cf7aef78c7bf7592059cf" exitCode=0
Nov 02 22:27:21.815854 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.815789    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" event={"ID":"818e9335-6d03-43c4-85a8-191d2ed5c6d1","Type":"ContainerDied","Data":"0782744162a2f8940060fe87bbf305f3d3e5c9f15f4cf7aef78c7bf7592059cf"}
Nov 02 22:27:21.917598 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.906507    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-cbgcz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 02 22:27:21.917598 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.906563    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 02 22:27:22.044460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.989602    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:22.044460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:21.989668    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:22.044460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.018459    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:22.298616 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.298488    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:22.298616 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.298555    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:22.442196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.357630    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:22.442196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.357692    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:22.442196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.358281    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 02 22:27:22.442196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.359206    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/health\": context deadline exceeded"
Nov 02 22:27:22.442196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.384537    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:22.456509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.456134    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:27:22.477780 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.477733    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:27:22.514390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.504465    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:22.514390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.504503    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": context deadline exceeded"
Nov 02 22:27:22.514390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.504572    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:22.514390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.504594    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:27:22.514390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.504956    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:22.514390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.505987    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:22.534389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.527003    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:22.548435 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.535468    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:22.548435 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.535522    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:22.619753 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.579609    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:22.620769 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.618906    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:22.621622 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.620313    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:22.674615 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.674581    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:27:22.732546 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.732487    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:22.757394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.756998    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-socket\") pod \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") "
Nov 02 22:27:22.757394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.757122    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-token\" (UniqueName: \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-token\") pod \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") "
Nov 02 22:27:22.757394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.757191    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-certs\") pod \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") "
Nov 02 22:27:22.757394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.757244    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-data\") pod \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") "
Nov 02 22:27:22.757394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.757274    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istiod-ca-cert\") pod \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") "
Nov 02 22:27:22.757394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.757321    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-podinfo\") pod \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") "
Nov 02 22:27:22.759134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.757978    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-credential-socket\") pod \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") "
Nov 02 22:27:22.759134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.758132    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-kube-api-access-vqwrg\" (UniqueName: \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-kube-api-access-vqwrg\") pod \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") "
Nov 02 22:27:22.759134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.758222    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-envoy\") pod \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\" (UID: \"818e9335-6d03-43c4-85a8-191d2ed5c6d1\") "
Nov 02 22:27:22.759134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.758748    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-data" pod "818e9335-6d03-43c4-85a8-191d2ed5c6d1" (UID: "818e9335-6d03-43c4-85a8-191d2ed5c6d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:27:22.759472 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.759143    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:27:22.760092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.760071    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-credential-socket" pod "818e9335-6d03-43c4-85a8-191d2ed5c6d1" (UID: "818e9335-6d03-43c4-85a8-191d2ed5c6d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:27:22.770387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.763784    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-socket" pod "818e9335-6d03-43c4-85a8-191d2ed5c6d1" (UID: "818e9335-6d03-43c4-85a8-191d2ed5c6d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:27:22.770812 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.770782    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istiod-ca-cert" pod "818e9335-6d03-43c4-85a8-191d2ed5c6d1" (UID: "818e9335-6d03-43c4-85a8-191d2ed5c6d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:27:22.782915 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.772717    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-certs" pod "818e9335-6d03-43c4-85a8-191d2ed5c6d1" (UID: "818e9335-6d03-43c4-85a8-191d2ed5c6d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:27:22.808704 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.808563    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-token" pod "818e9335-6d03-43c4-85a8-191d2ed5c6d1" (UID: "818e9335-6d03-43c4-85a8-191d2ed5c6d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:27:22.851390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.838568    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-kube-api-access-vqwrg" pod "818e9335-6d03-43c4-85a8-191d2ed5c6d1" (UID: "818e9335-6d03-43c4-85a8-191d2ed5c6d1"). InnerVolumeSpecName "kube-api-access-vqwrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:27:22.868395 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.863830    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:27:22.868395 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.863910    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:27:22.868395 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.863927    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:27:22.868395 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.863943    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqwrg\" (UniqueName: \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-kube-api-access-vqwrg\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:27:22.868395 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.863957    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:27:22.868395 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.863971    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:27:22.884556 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.884494    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-envoy" pod "818e9335-6d03-43c4-85a8-191d2ed5c6d1" (UID: "818e9335-6d03-43c4-85a8-191d2ed5c6d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:27:22.904404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.899336    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:22.904404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.899418    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:22.904841 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.904780    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-podinfo" pod "818e9335-6d03-43c4-85a8-191d2ed5c6d1" (UID: "818e9335-6d03-43c4-85a8-191d2ed5c6d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:27:23.050717 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:22.967899    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:27:23.145624 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.145515    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/818e9335-6d03-43c4-85a8-191d2ed5c6d1-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:27:23.145849 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.080792    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz" event={"ID":"818e9335-6d03-43c4-85a8-191d2ed5c6d1","Type":"ContainerDied","Data":"9460a62758af2a9f4bebd0d354151a3d93270d7d0d932101cadc79bbfb5a308e"}
Nov 02 22:27:23.149493 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.080894    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"
Nov 02 22:27:23.250124 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.250085    5067 scope.go:122] "RemoveContainer" containerID="0782744162a2f8940060fe87bbf305f3d3e5c9f15f4cf7aef78c7bf7592059cf"
Nov 02 22:27:23.275260 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.029686    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:23.316714 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.316676    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:23.334805 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.334760    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:23.450815 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.317432    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:23.451076 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.451039    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:23.451257 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.361147    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:23.451859 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.451836    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:27:23.456484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.456444    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:23.543656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.496778    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:23.543656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.496840    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:23.595386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.579935    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:23.595386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.580000    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:23.595386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.581809    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:23.595386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.582191    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:23.595386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.582530    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:27:23.595386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.582824    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:27:23.631413 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.596467    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:23.674388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.649052    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:23.675083 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.654469    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:23.692405 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.678566    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": context deadline exceeded"
Nov 02 22:27:23.696389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.654541    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:27:23.822825 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.821650    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:27:23.936587 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.887858    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:23.936587 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:23.887937    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:24.247561 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.247516    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"]
Nov 02 22:27:24.306391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.303193    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:24.312502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.312451    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:24.336393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.313528    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:24.415391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.415067    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:24.415391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.369576    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:24.415391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.327492    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:27:24.444631 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.422207    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-cbgcz"]
Nov 02 22:27:24.470454 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.460475    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:24.470454 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.461351    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 02 22:27:24.491960 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.491918    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:24.492228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.492197    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:24.526861 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.526760    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:24.527120 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.527088    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:24.590049 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.590005    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:24.590708 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.590677    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:24.591179 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.591153    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:24.592478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.592398    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:27:24.600501 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.600469    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:24.692648 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.692581    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:24.695402 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.692881    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:24.710487 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.710443    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:27:24.832452 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.832004    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:24.890272 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.890233    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:24.890588 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.890557    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:24.949902 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:24.949861    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:25.263350 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.263225    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:25.263350 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.263285    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:25.313067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.313022    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:25.313345 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.313313    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:25.416532 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.416482    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:27:25.461636 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.461591    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:25.462241 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.462211    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:27:25.589516 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.537515    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:25.589516 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.537576    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:25.589516 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.546693    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:25.589516 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.546769    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:25.609781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.608937    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:27:25.626549 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.626489    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:25.654394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.627234    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:25.654394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.628020    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:25.710417 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.697165    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:25.710417 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.697961    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:25.712774 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.711063    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:25.726389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.723123    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:25.902443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.901523    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:27:25.950815 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.950778    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" path="/var/lib/kubelet/pods/818e9335-6d03-43c4-85a8-191d2ed5c6d1/volumes"
Nov 02 22:27:25.997388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.970707    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:25.997388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:25.997073    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:26.299654 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.299616    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:26.300236 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.300203    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:26.319390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.313865    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:26.319390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.314011    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:26.319390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.314043    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:26.419833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.419784    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:26.478390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.473552    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:26.505295 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.505244    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:26.507174 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.507144    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 02 22:27:26.507430 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.507404    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 02 22:27:26.520408 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.520047    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:26.548764 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.548717    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:26.549053 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.548998    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:26.699473 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.697958    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:27:26.699473 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.698391    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:26.699473 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.698673    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:26.762389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.761872    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:26.763618 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.763582    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:27:26.764617 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.764589    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:26.765033 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.765005    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:26.790840 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.767278    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:27:26.889005 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.888969    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:26.913238 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.913196    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:26.914789 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.914746    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:26.914974 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:26.914950    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:27.267782 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.267742    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:27.294490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.283388    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:27.294910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.294770    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:27.311392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.308120    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:27.318430 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.309805    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:27.318700 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.318669    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:27.462484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.458768    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:27:27.522578 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.516517    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 02 22:27:27.526392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.522445    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:27.569808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.530838    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:27.586557 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.586517    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:27.586809 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.586778    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:27.673398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.672358    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": context deadline exceeded"
Nov 02 22:27:27.714587 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.714522    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:27.715258 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.715214    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:27:27.715443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.715420    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:27:27.787659 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.787548    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:27.837220 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.837179    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:27:27.837847 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.837810    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:27.872428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.859410    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:27.872428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.859754    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:27:27.955766 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:27.955714    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:28.009390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.001454    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 22:27:28.009390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.002344    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 22:27:28.348067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.348030    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:28.348645 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.348615    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:28.361983 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.361946    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:28.362234 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.362205    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:28.470391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.469461    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:27:28.536392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.531678    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:27:28.545596 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.544697    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:28.549389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.532781    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:28.549389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.547735    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:28.578647 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.578547    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:28.578930 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.578896    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:28.722703 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.722662    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:28.730397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.722622    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:27:28.743415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.740315    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:27:28.858616 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.858504    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:28.859182 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.859152    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:28.859593 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.859570    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:28.859932 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.859911    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:28.884969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:28.884917    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:29.009597 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.009551    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:27:29.262908 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.262866    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:29.280484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.280439    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:29.307418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.305348    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:29.307418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.305432    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:29.469797 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.469664    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:29.533258 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.532546    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:29.534128 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.534102    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:29.534339 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.534304    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:29.546262 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.545468    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:29.636033 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.635592    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:29.636033 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.635660    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:29.740211 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.739459    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:27:29.740211 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.739900    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:29.741773 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.741104    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:29.861657 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.861557    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:29.862640 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.862608    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:29.890241 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.862800    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:27:29.890683 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.862827    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:29.891021 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.888512    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:29.904391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.900819    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:29.908390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.904623    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": context deadline exceeded"
Nov 02 22:27:29.931637 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.931597    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:29.931928 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:29.931877    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.037591 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.037524    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.272837 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.272797    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:30.273144 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.273111    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:30.307805 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.307765    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 02 22:27:30.308099 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.308067    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 02 22:27:30.489191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.489145    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:27:30.518561 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.518464    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.535105 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.533714    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:27:30.542481 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.541951    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:30.542481 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.542013    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:30.566387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.556514    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.715392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.711466    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:30.715392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.711532    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.750394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.748893    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.750768 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.750736    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.751120 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.750894    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.867799 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.867528    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.902399 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.902249    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.924416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.908872    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.924416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.910044    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:30.924416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.911646    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:30.979390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.978490    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:30.979390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:30.978562    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.043391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.042447    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.055525 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.055469    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:27:31.298391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.291095    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 22:27:31.313389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.298665    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 22:27:31.501390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.500521    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.538495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.538425    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.542876 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.542843    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 22:27:31.543107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.543077    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 22:27:31.563514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.562842    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.752391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.751921    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.752602 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.752536    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:27:31.759794 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.759439    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:27:31.760478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.760445    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:31.764477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.764443    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.876083 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.875978    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.913673 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.913630    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:31.914319 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.914283    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.916007 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.914540    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.932755 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.931974    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:31.988015 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.987568    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:31.988015 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:31.987641    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.058307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.057472    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:27:32.294940 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.294895    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/health\": context deadline exceeded"
Nov 02 22:27:32.358548 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.358511    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:32.358959 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.358904    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.389795 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.389738    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.487390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.480495    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:32.487390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.480590    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.522392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.503447    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.562586 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.554794    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.582389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.582216    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.589794 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.588933    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 02 22:27:32.756529 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.756467    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.757098 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.757067    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:32.785936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.785892    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:27:32.836569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.836476    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:32.836824 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.836791    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.877309 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.877266    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.917503 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.917457    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:27:32.918163 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.918123    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:32.919499 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.919473    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:32.933787 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:32.933749    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.008391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.000488    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:33.008391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.000557    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.059502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.059453    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.323887 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.322522    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:33.324192 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.324138    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.365386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.358945    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:27:33.403515 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.403476    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:33.403779 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.403746    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.616450 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.599052    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.695401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.631570    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:27:33.695401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.649400    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:33.695401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.649455    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.697557 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.697516    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": context deadline exceeded"
Nov 02 22:27:33.787436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.757202    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.787436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.757571    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:27:33.792658 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.792617    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:27:33.890449 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.889920    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.892423 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.890824    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:27:33.920521 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.920480    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.921164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.921126    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:27:33.921664 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.921641    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:33.930335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.929950    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:33.930335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.930003    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.939774 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.937431    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:33.970415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.969761    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:33.975631 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:33.975606    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:27:34.016046 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.016012    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:34.048509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.048466    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:34.048755 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.048730    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:34.092064 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.092016    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:34.104036 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.104001    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:27:34.367506 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.367451    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:27:34.373388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.368127    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:34.417344 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.417302    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:34.417616 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.417584    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:34.558170 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.556916    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:27:34.626479 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.612430    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:34.760950 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.760895    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:34.772790 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.772751    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:34.812542 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.807801    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:34.826808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.808754    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:34.865684 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.844039    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:27:34.891576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.891479    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:27:34.915617 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.915586    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:27:34.965382 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.936173    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:27:34.965959 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.948855    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:27:34.972247 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.948889    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:27:34.983719 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.949698    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:34.984405 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:34.949742    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:35.117389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.103444    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:35.140383 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.135020    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:27:35.161440 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.159731    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:27:35.183726 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.183694    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:27:35.277062 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.277025    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:27:35.369576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.367832    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:35.369576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.368226    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:27:35.475941 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.475523    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:35.475941 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.475589    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:35.652516 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.651524    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:27:35.652516 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.651606    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:27:35.681944 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.680463    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:27:35.681944 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.681685    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:35.800449 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:35.765482    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:27:36.033158 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:36.033090    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:27:36.128538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:36.128443    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:27:36.639291 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:36.639257    5067 patch_prober.go:29] interesting pod/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:27:36.639588 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:36.639558    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:27:37.388677 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:37.388642    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:37.404644 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:37.404611    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:27:37.588890 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:27:37.588825    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:27:40.783542 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.783506    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"]
Nov 02 22:27:40.784136 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.784123    5067 cpu_manager.go:414] "containerMap: removing container" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy"
Nov 02 22:27:40.784182 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.784138    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy"
Nov 02 22:27:40.784298 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.784275    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="818e9335-6d03-43c4-85a8-191d2ed5c6d1" containerName="istio-proxy"
Nov 02 22:27:40.788188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.788171    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.842491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.842458    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-istio-token\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.842674 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.842595    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a56fb386-b822-4107-b8ed-11c593c9a268-istiod-ca-cert\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.842674 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.842641    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-envoy\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.842674 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.842664    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-data\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.842838 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.842811    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-certs\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.842916 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.842888    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cpgb\" (UniqueName: \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-kube-api-access-2cpgb\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.843066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.842947    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-credential-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.843066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.843029    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a56fb386-b822-4107-b8ed-11c593c9a268-istio-podinfo\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.843185 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.843159    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.845096 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.845076    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"]
Nov 02 22:27:40.944959 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.944923    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-istio-token\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945022    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a56fb386-b822-4107-b8ed-11c593c9a268-istiod-ca-cert\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945085    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-envoy\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945175 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945119    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-data\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945175    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-certs\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945236    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cpgb\" (UniqueName: \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-kube-api-access-2cpgb\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945306    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-credential-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945407    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a56fb386-b822-4107-b8ed-11c593c9a268-istio-podinfo\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945529    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945651 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945607    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-data\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945744 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945725    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-credential-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945881 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945860    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a56fb386-b822-4107-b8ed-11c593c9a268-istiod-ca-cert\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945951 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945918    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-socket\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.945998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.945985    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-certs\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.948263 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.948235    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-envoy\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:40.948428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:40.948396    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a56fb386-b822-4107-b8ed-11c593c9a268-istio-podinfo\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:41.003060 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:41.003028    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-istio-token\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:41.006852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:41.006831    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cpgb\" (UniqueName: \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-kube-api-access-2cpgb\") pod \"e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") " pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:41.102952 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:41.102870    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:41.530347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:41.530309    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"]
Nov 02 22:27:41.565078 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:27:41.565046    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56fb386_b822_4107_b8ed_11c593c9a268.slice/crio-8341cb62523b8b25826f3be6cd8f8824a9ac8816d06b917a92b76d0f327d710d WatchSource:0}: Error finding container 8341cb62523b8b25826f3be6cd8f8824a9ac8816d06b917a92b76d0f327d710d: Status 404 returned error can't find the container with id 8341cb62523b8b25826f3be6cd8f8824a9ac8816d06b917a92b76d0f327d710d
Nov 02 22:27:41.567700 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:41.567659    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:41.567786 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:41.567757    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:41.567852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:41.567800    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:27:42.177454 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:42.177417    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" event={"ID":"a56fb386-b822-4107-b8ed-11c593c9a268","Type":"ContainerStarted","Data":"b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89"}
Nov 02 22:27:42.178004 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:42.177479    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" event={"ID":"a56fb386-b822-4107-b8ed-11c593c9a268","Type":"ContainerStarted","Data":"8341cb62523b8b25826f3be6cd8f8824a9ac8816d06b917a92b76d0f327d710d"}
Nov 02 22:27:42.363966 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:42.363902    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" podStartSLOduration=2.363883344 podStartE2EDuration="2.363883344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:27:42.356871543 +0000 UTC m=+1334.640681268" watchObservedRunningTime="2026-11-02 22:27:42.363883344 +0000 UTC m=+1334.647693028"
Nov 02 22:27:43.103448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:43.103412    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:43.105683 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:43.105654    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 02 22:27:43.105826 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:43.105717    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 02 22:27:44.103736 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:44.103695    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 02 22:27:44.106182 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:44.103770    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 02 22:27:45.104827 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:45.104792    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 02 22:27:45.105237 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:45.104861    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 02 22:27:46.103675 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:46.103635    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 02 22:27:46.103974 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:46.103943    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 02 22:27:47.104397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:47.104346    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 02 22:27:47.104788 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:47.104423    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 02 22:27:48.103485 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:48.103439    5067 patch_prober.go:29] interesting pod/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 02 22:27:48.103693 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:48.103514    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 02 22:27:49.107717 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:49.107681    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:49.108119 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:49.108035    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:49.108741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:49.108721    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:27:57.066721 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:57.066669    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7216-49cac6"]
Nov 02 22:27:57.072105 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:57.072077    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:27:57.121851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:57.121820    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7216-49cac6"]
Nov 02 22:27:57.160987 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:57.160949    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8tbn\" (UniqueName: \"kubernetes.io/projected/c226720c-0f3d-43e6-9502-f6d7b6046288-kube-api-access-m8tbn\") pod \"test-curl-7216-49cac6\" (UID: \"c226720c-0f3d-43e6-9502-f6d7b6046288\") " pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:27:57.162883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:57.162856    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-thb7m\""
Nov 02 22:27:57.262494 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:57.262450    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8tbn\" (UniqueName: \"kubernetes.io/projected/c226720c-0f3d-43e6-9502-f6d7b6046288-kube-api-access-m8tbn\") pod \"test-curl-7216-49cac6\" (UID: \"c226720c-0f3d-43e6-9502-f6d7b6046288\") " pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:27:57.320143 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:57.320067    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8tbn\" (UniqueName: \"kubernetes.io/projected/c226720c-0f3d-43e6-9502-f6d7b6046288-kube-api-access-m8tbn\") pod \"test-curl-7216-49cac6\" (UID: \"c226720c-0f3d-43e6-9502-f6d7b6046288\") " pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:27:57.385332 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:57.385295    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:27:57.761618 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:57.761584    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7216-49cac6"]
Nov 02 22:27:57.782948 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:27:57.782870    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc226720c_0f3d_43e6_9502_f6d7b6046288.slice/crio-75a7b7f3430ed1895316030fc5d5cddc74901d8b627131841dcceddad1869f9d WatchSource:0}: Error finding container 75a7b7f3430ed1895316030fc5d5cddc74901d8b627131841dcceddad1869f9d: Status 404 returned error can't find the container with id 75a7b7f3430ed1895316030fc5d5cddc74901d8b627131841dcceddad1869f9d
Nov 02 22:27:57.784561 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:57.784544    5067 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:27:58.300438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:27:58.300400    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7216-49cac6" event={"ID":"c226720c-0f3d-43e6-9502-f6d7b6046288","Type":"ContainerStarted","Data":"75a7b7f3430ed1895316030fc5d5cddc74901d8b627131841dcceddad1869f9d"}
Nov 02 22:28:00.551824 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:00.551781    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"]
Nov 02 22:28:00.552942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:00.552621    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy" containerID="cri-o://763cd89501003e72ec6c2a04e41db46a182beec439ca10181a71111be1c0c93c" gracePeriod=30
Nov 02 22:28:01.322736 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:01.322690    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7216-49cac6" event={"ID":"c226720c-0f3d-43e6-9502-f6d7b6046288","Type":"ContainerStarted","Data":"6a1aa9d1c55aa9b5a0c6b92176d6d3a468b790fb2620781971c6746071727a7f"}
Nov 02 22:28:01.430903 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:01.430849    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7216-49cac6" podStartSLOduration=1.531779185 podStartE2EDuration="4.430834453s" totalImagesPullingTime="2.899055268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:28:01.429769435 +0000 UTC m=+1353.713579143" watchObservedRunningTime="2026-11-02 22:28:01.430834453 +0000 UTC m=+1353.714644195"
Nov 02 22:28:02.973681 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:02.973646    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7216-49cac6"]
Nov 02 22:28:02.978460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:02.978426    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7216-49cac6"]
Nov 02 22:28:03.340859 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:03.340753    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7216-49cac6" podUID="c226720c-0f3d-43e6-9502-f6d7b6046288" containerName="test-curl-7216-49cac6" containerID="cri-o://6a1aa9d1c55aa9b5a0c6b92176d6d3a468b790fb2620781971c6746071727a7f" gracePeriod=2
Nov 02 22:28:03.402411 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:03.392643    5067 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7216-49cac6\" is forbidden: User \"system:node:ip-10-0-130-41.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-41.us-west-2.compute.internal' and this object" podUID="c226720c-0f3d-43e6-9502-f6d7b6046288" pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:28:04.558715 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.557151    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"]
Nov 02 22:28:04.589311 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.589261    5067 cpu_manager.go:414] "containerMap: removing container" podUID="c226720c-0f3d-43e6-9502-f6d7b6046288" containerName="test-curl-7216-49cac6"
Nov 02 22:28:04.618682 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.593398    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c226720c-0f3d-43e6-9502-f6d7b6046288" containerName="test-curl-7216-49cac6"
Nov 02 22:28:04.685769 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.685715    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="c226720c-0f3d-43e6-9502-f6d7b6046288" containerName="test-curl-7216-49cac6"
Nov 02 22:28:04.837197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.837115    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"]
Nov 02 22:28:04.837552 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.837530    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:04.919418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.876607    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-credential-socket\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:04.919418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.876656    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x959x\" (UniqueName: \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-kube-api-access-x959x\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:04.919418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.876735    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-envoy\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:04.919418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.876762    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-socket\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:04.919418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.876795    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-certs\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:04.919418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.876840    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8881e503-831c-4519-b1b1-75312e378a79-istio-podinfo\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:04.919418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.876935    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8881e503-831c-4519-b1b1-75312e378a79-istiod-ca-cert\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:04.919418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.876975    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-data\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:04.919418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:04.877013    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-istio-token\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.004390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.003875    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2e86dfb1-openshift-default-dockercfg-4w4ff\""
Nov 02 22:28:05.065489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.046501    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-credential-socket\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.065489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.046551    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x959x\" (UniqueName: \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-kube-api-access-x959x\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.065489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.046639    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-envoy\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.065489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.046666    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-socket\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.065489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.046703    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-certs\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.065489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.046752    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8881e503-831c-4519-b1b1-75312e378a79-istio-podinfo\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.065489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.046856    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8881e503-831c-4519-b1b1-75312e378a79-istiod-ca-cert\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.065489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.046898    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-data\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.065489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.046934    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-istio-token\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.065489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.047857    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-credential-socket\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.066721 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.066693    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-certs\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.082484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.067453    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-socket\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.153236 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.094731    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8881e503-831c-4519-b1b1-75312e378a79-istiod-ca-cert\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.153236 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.096153    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:05.153236 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.106966    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-envoy\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.153236 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.146092    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:05.153236 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.146746    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-istio-token\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.153236 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.146834    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-data\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.194389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.190514    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8881e503-831c-4519-b1b1-75312e378a79-istio-podinfo\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.194389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.193301    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x959x\" (UniqueName: \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-kube-api-access-x959x\") pod \"e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") " pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.196231 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.194693    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:28:05.383924 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.383886    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:05.464537 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:05.464495    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:06.259701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.215482    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:06.259701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.216070    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:06.259701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.220667    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:06.412389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.411538    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:06.470389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.461481    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:28:06.588847 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.588809    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:28:06.589140 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.589122    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:28:06.600395 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.591157    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:28:06.664405 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.662001    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"]
Nov 02 22:28:06.772004 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.763737    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:28:06.772004 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.763832    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:28:06.772004 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.763878    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:28:06.783067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.783029    5067 generic.go:356] "Generic (PLEG): container finished" podID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerID="763cd89501003e72ec6c2a04e41db46a182beec439ca10181a71111be1c0c93c" exitCode=0
Nov 02 22:28:06.783644 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.783317    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" event={"ID":"0ddee653-e8f0-42a6-9c14-3361ccc6dc97","Type":"ContainerDied","Data":"763cd89501003e72ec6c2a04e41db46a182beec439ca10181a71111be1c0c93c"}
Nov 02 22:28:06.791728 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.791700    5067 scope.go:122] "RemoveContainer" containerID="f3432dddba01ac5a6cabae65d3db995397765409a6110439e0c85c9336e0ed9e"
Nov 02 22:28:06.799469 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.799437    5067 generic.go:356] "Generic (PLEG): container finished" podID="c226720c-0f3d-43e6-9502-f6d7b6046288" containerID="6a1aa9d1c55aa9b5a0c6b92176d6d3a468b790fb2620781971c6746071727a7f" exitCode=137
Nov 02 22:28:06.957195 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.957098    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:28:06.990704 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:06.989502    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:28:07.101401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.093592    5067 patch_prober.go:29] interesting pod/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:28:07.101401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.098484    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:07.157953 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.156451    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:07.234801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.219449    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:28:07.234801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.229908    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:07.234801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.230934    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:07.234801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.231307    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:28:07.471390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.467577    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:07.498484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.446550    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:07.498484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.477917    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:28:07.498484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.484074    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:28:07.576887 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.574723    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:07.608824 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.601156    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:28:07.608824 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.602975    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:28:07.957445 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:07.918713    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:28:08.025291 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:07.959959    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:28:08.025781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.005710    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:08.277012 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.276956    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:08.277522 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.277487    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:08.325422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.312458    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:08.351441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.334840    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:08.370146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.370113    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:28:08.482560 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.482517    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:28:08.525401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.523460    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:08.571559 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.571455    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:08.587441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.587407    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-envoy\") pod \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") "
Nov 02 22:28:08.587829 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.587802    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-token\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-token\") pod \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") "
Nov 02 22:28:08.587980 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.587966    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-kube-api-access-rgcvj\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-kube-api-access-rgcvj\") pod \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") "
Nov 02 22:28:08.588149 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.588134    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istiod-ca-cert\") pod \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") "
Nov 02 22:28:08.596491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.596459    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-credential-socket\") pod \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") "
Nov 02 22:28:08.596756 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.596735    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-certs\") pod \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") "
Nov 02 22:28:08.596919 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.596904    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-podinfo\") pod \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") "
Nov 02 22:28:08.636297 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.636257    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-socket\") pod \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") "
Nov 02 22:28:08.636612 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.636590    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-data\") pod \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") "
Nov 02 22:28:08.637828 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.604873    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:08.638951 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.605684    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-credential-socket" pod "0ddee653-e8f0-42a6-9c14-3361ccc6dc97" (UID: "0ddee653-e8f0-42a6-9c14-3361ccc6dc97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:08.639098 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.605908    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-certs" pod "0ddee653-e8f0-42a6-9c14-3361ccc6dc97" (UID: "0ddee653-e8f0-42a6-9c14-3361ccc6dc97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:08.639191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.625713    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istiod-ca-cert" pod "0ddee653-e8f0-42a6-9c14-3361ccc6dc97" (UID: "0ddee653-e8f0-42a6-9c14-3361ccc6dc97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:28:08.639278 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.637801    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-data" pod "0ddee653-e8f0-42a6-9c14-3361ccc6dc97" (UID: "0ddee653-e8f0-42a6-9c14-3361ccc6dc97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:08.639352 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.638926    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-socket" pod "0ddee653-e8f0-42a6-9c14-3361ccc6dc97" (UID: "0ddee653-e8f0-42a6-9c14-3361ccc6dc97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:08.874609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.873524    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-envoy" pod "0ddee653-e8f0-42a6-9c14-3361ccc6dc97" (UID: "0ddee653-e8f0-42a6-9c14-3361ccc6dc97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:08.874609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.873652    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-token" pod "0ddee653-e8f0-42a6-9c14-3361ccc6dc97" (UID: "0ddee653-e8f0-42a6-9c14-3361ccc6dc97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:28:08.874609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.873709    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-podinfo" pod "0ddee653-e8f0-42a6-9c14-3361ccc6dc97" (UID: "0ddee653-e8f0-42a6-9c14-3361ccc6dc97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:28:08.876540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.876310    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:08.876540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.876345    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:08.876540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.876385    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:08.876540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.876400    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:08.876540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.876414    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:08.876540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.876427    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:08.876540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.876439    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:08.876540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.876452    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:08.979641 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:08.979602    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-kube-api-access-rgcvj" pod "0ddee653-e8f0-42a6-9c14-3361ccc6dc97" (UID: "0ddee653-e8f0-42a6-9c14-3361ccc6dc97"). InnerVolumeSpecName "kube-api-access-rgcvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:28:09.032435 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.016503    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-kube-api-access-rgcvj\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-kube-api-access-rgcvj\") pod \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\" (UID: \"0ddee653-e8f0-42a6-9c14-3361ccc6dc97\") "
Nov 02 22:28:09.045425 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:28:09.044994    5067 empty_dir.go:505] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/0ddee653-e8f0-42a6-9c14-3361ccc6dc97/volumes/kubernetes.io~projected/kube-api-access-rgcvj
Nov 02 22:28:09.045425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.045164    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-kube-api-access-rgcvj" pod "0ddee653-e8f0-42a6-9c14-3361ccc6dc97" (UID: "0ddee653-e8f0-42a6-9c14-3361ccc6dc97"). InnerVolumeSpecName "kube-api-access-rgcvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:28:09.133226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.126481    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgcvj\" (UniqueName: \"kubernetes.io/projected/0ddee653-e8f0-42a6-9c14-3361ccc6dc97-kube-api-access-rgcvj\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:09.177499 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.177466    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"
Nov 02 22:28:09.356391 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:09.349090    5067 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.491s"
Nov 02 22:28:09.356391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.349181    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:28:09.356391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.349328    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:28:09.356391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.349347    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7" event={"ID":"0ddee653-e8f0-42a6-9c14-3361ccc6dc97","Type":"ContainerDied","Data":"dea50c39ec90f665443547f22454a5df1281725d09aaaa9f10d2323d0acaf176"}
Nov 02 22:28:09.356391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.349436    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:28:09.356391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.349478    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:28:09.356391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.349513    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:28:09.358605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.358578    5067 scope.go:122] "RemoveContainer" containerID="763cd89501003e72ec6c2a04e41db46a182beec439ca10181a71111be1c0c93c"
Nov 02 22:28:09.418728 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.378197    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:09.420402 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.378236    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 22:28:09.421030 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.378272    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:09.421420 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.378291    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Nov 02 22:28:09.421755 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.378310    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 02 22:28:09.440237 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.378328    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:28:09.440237 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.399439    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" event={"ID":"8881e503-831c-4519-b1b1-75312e378a79","Type":"ContainerStarted","Data":"1c5c859b792be5da9fb52c45fea6a4ccb7eb5fc846b5c30ffc8c8b4414261a81"}
Nov 02 22:28:09.528692 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.528642    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:09.530221 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.530173    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:09.646394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.550454    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:09.646394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.577197    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:09.646394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.631825    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:28:09.647263 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.646808    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:09.915537 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.915486    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:09.949580 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:09.920509    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.533723 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.531749    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.533723 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.531996    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.533723 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.532386    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.533723 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.532573    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.533723 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.532953    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.533723 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.533148    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.533723 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.533475    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.534668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.534636    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.535969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.535938    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.579606 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.556950    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:28:10.579606 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.557259    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:28:10.579606 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.579591    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:28:10.579969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.579713    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.632594 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.632541    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.651388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.633964    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:28:10.695537 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.695480    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:10.905447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:10.880149    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c226720c-0f3d-43e6-9502-f6d7b6046288-kube-api-access-m8tbn\" (UniqueName: \"kubernetes.io/projected/c226720c-0f3d-43e6-9502-f6d7b6046288-kube-api-access-m8tbn\") pod \"c226720c-0f3d-43e6-9502-f6d7b6046288\" (UID: \"c226720c-0f3d-43e6-9502-f6d7b6046288\") "
Nov 02 22:28:11.053389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.040465    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"]
Nov 02 22:28:11.053389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.052511    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" event={"ID":"8881e503-831c-4519-b1b1-75312e378a79","Type":"ContainerStarted","Data":"e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047"}
Nov 02 22:28:11.053389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.052573    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-7sfm7"]
Nov 02 22:28:11.053389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.050876    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:11.128637 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.128604    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:28:11.148405 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.128674    5067 scope.go:122] "RemoveContainer" containerID="6a1aa9d1c55aa9b5a0c6b92176d6d3a468b790fb2620781971c6746071727a7f"
Nov 02 22:28:11.197470 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:11.197275    5067 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7216-49cac6\" is forbidden: User \"system:node:ip-10-0-130-41.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-41.us-west-2.compute.internal' and this object" podUID="c226720c-0f3d-43e6-9502-f6d7b6046288" pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:28:11.255710 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.219291    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c226720c-0f3d-43e6-9502-f6d7b6046288-kube-api-access-m8tbn" pod "c226720c-0f3d-43e6-9502-f6d7b6046288" (UID: "c226720c-0f3d-43e6-9502-f6d7b6046288"). InnerVolumeSpecName "kube-api-access-m8tbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:28:11.255710 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.242839    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8tbn\" (UniqueName: \"kubernetes.io/projected/c226720c-0f3d-43e6-9502-f6d7b6046288-kube-api-access-m8tbn\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:11.414768 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.414733    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:11.588404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.576550    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded"
Nov 02 22:28:11.820344 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:11.591763    5067 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7216-49cac6\" is forbidden: User \"system:node:ip-10-0-130-41.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-41.us-west-2.compute.internal' and this object" podUID="c226720c-0f3d-43e6-9502-f6d7b6046288" pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:28:11.820344 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.593060    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:11.820344 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.659801    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 02 22:28:11.840569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.792669    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:28:11.891412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.842477    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:11.891412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.843525    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:11.891412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:11.866762    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 02 22:28:12.004748 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.004688    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:28:12.005702 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.005667    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 02 22:28:12.103497 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.103407    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 02 22:28:12.106999 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.106967    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 02 22:28:12.124690 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.009663    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:12.141387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.134876    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:12.210832 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:12.210783    5067 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7216-49cac6\" is forbidden: User \"system:node:ip-10-0-130-41.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-41.us-west-2.compute.internal' and this object" podUID="c226720c-0f3d-43e6-9502-f6d7b6046288" pod="opendatahub/test-curl-7216-49cac6"
Nov 02 22:28:12.250247 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.250207    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" path="/var/lib/kubelet/pods/0ddee653-e8f0-42a6-9c14-3361ccc6dc97/volumes"
Nov 02 22:28:12.251164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.251142    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c226720c-0f3d-43e6-9502-f6d7b6046288" path="/var/lib/kubelet/pods/c226720c-0f3d-43e6-9502-f6d7b6046288/volumes"
Nov 02 22:28:12.253630 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.253590    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" podStartSLOduration=8.253576563 podStartE2EDuration="8.253576563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:28:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:28:12.154315668 +0000 UTC m=+1364.438125351" watchObservedRunningTime="2026-11-02 22:28:12.253576563 +0000 UTC m=+1364.537386250"
Nov 02 22:28:12.255425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.255401    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:28:12.259308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.255509    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:28:12.259308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.255614    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:28:12.259308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.255668    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:28:12.259308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.255738    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:28:12.259308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.255795    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:28:12.259308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.255837    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:28:12.259308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.255876    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:28:12.259308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.255934    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:28:12.259308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.255982    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:28:12.269773 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.269463    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:28:12.270912 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.270875    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:28:12.282399 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.281594    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:12.284640 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:12.284620    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:28:21.209414 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:21.209339    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"]
Nov 02 22:28:21.255824 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:21.255784    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"]
Nov 02 22:28:21.260401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:21.260374    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"]
Nov 02 22:28:26.912154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.912097    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"]
Nov 02 22:28:26.913676 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.913646    5067 cpu_manager.go:414] "containerMap: removing container" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy"
Nov 02 22:28:26.913676 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.913676    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy"
Nov 02 22:28:26.913857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.913723    5067 cpu_manager.go:414] "containerMap: removing container" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy"
Nov 02 22:28:26.913857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.913731    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy"
Nov 02 22:28:26.922099 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.922058    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy"
Nov 02 22:28:26.925656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.925636    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:26.970604 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.970562    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"]
Nov 02 22:28:26.975610 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.975582    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7f1542e5-serving-cert\""
Nov 02 22:28:26.987797 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.987757    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-maas-api-tls\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:26.987974 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.987859    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-metrics-certs\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:26.987974 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.987893    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2z4n\" (UniqueName: \"kubernetes.io/projected/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-kube-api-access-s2z4n\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:26.988095 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:26.988019    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-openshift-service-ca\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:27.089988 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:27.089953    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-maas-api-tls\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:27.090161 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:27.090012    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-metrics-certs\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:27.090161 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:27.090029    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2z4n\" (UniqueName: \"kubernetes.io/projected/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-kube-api-access-s2z4n\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:27.090161 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:27.090081    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-openshift-service-ca\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:27.090161 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:27.090099    5067 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-serving-cert: secret "maas-api-e2e-ait-7f1542e5-serving-cert" not found
Nov 02 22:28:27.090161 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:27.090161    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-maas-api-tls podName:dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0 nodeName:}" failed. No retries permitted until 2026-11-02 22:28:27.590143951 +0000 UTC m=+1379.873953616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-maas-api-tls") pod "maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n" (UID: "dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0") : secret "maas-api-e2e-ait-7f1542e5-serving-cert" not found
Nov 02 22:28:27.090796 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:27.090776    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-openshift-service-ca\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:27.092792 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:27.092771    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-metrics-certs\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:27.150280 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:27.150242    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2z4n\" (UniqueName: \"kubernetes.io/projected/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-kube-api-access-s2z4n\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:27.599936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:27.599898    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-maas-api-tls\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:27.602351 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:27.602326    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-maas-api-tls\") pod \"maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:27.843549 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:27.843518    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:28.223088 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:28.223061    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"]
Nov 02 22:28:28.224932 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:28:28.224905    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbfd9b4a_d6cc_45d8_aa9b_dbe56dad9bb0.slice/crio-8f4fbc9fca6fa29464b5f91a5a16f280cadf8a6252ba25a1f791812fc3c3603e WatchSource:0}: Error finding container 8f4fbc9fca6fa29464b5f91a5a16f280cadf8a6252ba25a1f791812fc3c3603e: Status 404 returned error can't find the container with id 8f4fbc9fca6fa29464b5f91a5a16f280cadf8a6252ba25a1f791812fc3c3603e
Nov 02 22:28:28.401024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:28.400996    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n" event={"ID":"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0","Type":"ContainerStarted","Data":"8f4fbc9fca6fa29464b5f91a5a16f280cadf8a6252ba25a1f791812fc3c3603e"}
Nov 02 22:28:29.409680 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:29.409575    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n" event={"ID":"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0","Type":"ContainerStarted","Data":"5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6"}
Nov 02 22:28:29.409680 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:29.409666    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:29.523104 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:29.522963    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n" podStartSLOduration=2.604631977 podStartE2EDuration="3.52294293s" totalImagesPullingTime="918.310953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:28:29.519674486 +0000 UTC m=+1381.803484197" watchObservedRunningTime="2026-11-02 22:28:29.52294293 +0000 UTC m=+1381.806752618"
Nov 02 22:28:35.422135 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:35.422105    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:28:36.231427 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.231338    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main" containerID="cri-o://e86f37a930f7ebe0b74347037fbcf5065f5bfd12c2b9bedabc9f7ddd0109714a" gracePeriod=45
Nov 02 22:28:36.277861 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.277800    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" containerID="cri-o://bdcd1a4cbb8ebd728f2ddee03b7942e7ee4a9636527bf14f1fd375719ed3ad6f" gracePeriod=45
Nov 02 22:28:36.281172 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.281140    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" containerID="cri-o://a025d17782de4732d4cb9150a5a3ff3b2ce46f611de207df641583e86a51c297" gracePeriod=45
Nov 02 22:28:36.313098 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.313050    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": dial tcp 10.132.0.60:8000: connect: connection refused"
Nov 02 22:28:36.514462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.514334    5067 generic.go:356] "Generic (PLEG): container finished" podID="e83f7463-2538-40cb-a284-ee2179b6d690" containerID="a025d17782de4732d4cb9150a5a3ff3b2ce46f611de207df641583e86a51c297" exitCode=0
Nov 02 22:28:36.515019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.514396    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" event={"ID":"e83f7463-2538-40cb-a284-ee2179b6d690","Type":"ContainerDied","Data":"a025d17782de4732d4cb9150a5a3ff3b2ce46f611de207df641583e86a51c297"}
Nov 02 22:28:36.520933 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.520901    5067 generic.go:356] "Generic (PLEG): container finished" podID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerID="e86f37a930f7ebe0b74347037fbcf5065f5bfd12c2b9bedabc9f7ddd0109714a" exitCode=0
Nov 02 22:28:36.521095 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.520968    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" event={"ID":"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5","Type":"ContainerDied","Data":"e86f37a930f7ebe0b74347037fbcf5065f5bfd12c2b9bedabc9f7ddd0109714a"}
Nov 02 22:28:36.522632 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.522583    5067 generic.go:356] "Generic (PLEG): container finished" podID="02407f93-fd96-49c9-a18c-348651482119" containerID="bdcd1a4cbb8ebd728f2ddee03b7942e7ee4a9636527bf14f1fd375719ed3ad6f" exitCode=0
Nov 02 22:28:36.522746 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.522639    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" event={"ID":"02407f93-fd96-49c9-a18c-348651482119","Type":"ContainerDied","Data":"bdcd1a4cbb8ebd728f2ddee03b7942e7ee4a9636527bf14f1fd375719ed3ad6f"}
Nov 02 22:28:36.632900 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.632867    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:28:36.637644 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.637621    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:28:36.663616 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.663556    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:28:36.687766 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.687729    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-model-cache\") pod \"e83f7463-2538-40cb-a284-ee2179b6d690\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") "
Nov 02 22:28:36.687980 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.687845    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-home\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-home\") pod \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") "
Nov 02 22:28:36.687980 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.687976    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tmp-dir\") pod \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") "
Nov 02 22:28:36.688112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688065    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e83f7463-2538-40cb-a284-ee2179b6d690-tls-certs\" (UniqueName: \"kubernetes.io/secret/e83f7463-2538-40cb-a284-ee2179b6d690-tls-certs\") pod \"e83f7463-2538-40cb-a284-ee2179b6d690\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") "
Nov 02 22:28:36.688112 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688094    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-dshm\") pod \"e83f7463-2538-40cb-a284-ee2179b6d690\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") "
Nov 02 22:28:36.688279 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688125    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-model-cache\") pod \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") "
Nov 02 22:28:36.688279 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688187    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-kube-api-access-m9xlq\" (UniqueName: \"kubernetes.io/projected/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-kube-api-access-m9xlq\") pod \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") "
Nov 02 22:28:36.688279 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688234    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-tmp-dir\") pod \"e83f7463-2538-40cb-a284-ee2179b6d690\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") "
Nov 02 22:28:36.688279 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688262    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e83f7463-2538-40cb-a284-ee2179b6d690-kube-api-access-tr8pn\" (UniqueName: \"kubernetes.io/projected/e83f7463-2538-40cb-a284-ee2179b6d690-kube-api-access-tr8pn\") pod \"e83f7463-2538-40cb-a284-ee2179b6d690\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") "
Nov 02 22:28:36.690478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688305    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tls-certs\" (UniqueName: \"kubernetes.io/secret/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tls-certs\") pod \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") "
Nov 02 22:28:36.690478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688352    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-home\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-home\") pod \"e83f7463-2538-40cb-a284-ee2179b6d690\" (UID: \"e83f7463-2538-40cb-a284-ee2179b6d690\") "
Nov 02 22:28:36.690478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688413    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-dshm\") pod \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\" (UID: \"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5\") "
Nov 02 22:28:36.690478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688677    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-model-cache" pod "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" (UID: "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.690478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.689159    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.690478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.688887    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-model-cache" pod "e83f7463-2538-40cb-a284-ee2179b6d690" (UID: "e83f7463-2538-40cb-a284-ee2179b6d690"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.690478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.689776    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-home" pod "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" (UID: "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.690478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.689956    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tmp-dir" pod "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" (UID: "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.691985 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.691950    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-tmp-dir" pod "e83f7463-2538-40cb-a284-ee2179b6d690" (UID: "e83f7463-2538-40cb-a284-ee2179b6d690"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.692167 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.692138    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-home" pod "e83f7463-2538-40cb-a284-ee2179b6d690" (UID: "e83f7463-2538-40cb-a284-ee2179b6d690"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.705837 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.705795    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-dshm" pod "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" (UID: "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.712873 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.712823    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-dshm" pod "e83f7463-2538-40cb-a284-ee2179b6d690" (UID: "e83f7463-2538-40cb-a284-ee2179b6d690"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.713157 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.713125    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e83f7463-2538-40cb-a284-ee2179b6d690-tls-certs" pod "e83f7463-2538-40cb-a284-ee2179b6d690" (UID: "e83f7463-2538-40cb-a284-ee2179b6d690"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:28:36.713412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.713351    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tls-certs" pod "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" (UID: "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:28:36.713695 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.713665    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-kube-api-access-m9xlq" pod "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" (UID: "eaad5c27-f022-46cd-9f1a-a8f3390fe0a5"). InnerVolumeSpecName "kube-api-access-m9xlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:28:36.715585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.715548    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e83f7463-2538-40cb-a284-ee2179b6d690-kube-api-access-tr8pn" pod "e83f7463-2538-40cb-a284-ee2179b6d690" (UID: "e83f7463-2538-40cb-a284-ee2179b6d690"). InnerVolumeSpecName "kube-api-access-tr8pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:28:36.790980 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.790867    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/02407f93-fd96-49c9-a18c-348651482119-tls-certs\" (UniqueName: \"kubernetes.io/secret/02407f93-fd96-49c9-a18c-348651482119-tls-certs\") pod \"02407f93-fd96-49c9-a18c-348651482119\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") "
Nov 02 22:28:36.790980 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.790944    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-tmp-dir\") pod \"02407f93-fd96-49c9-a18c-348651482119\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") "
Nov 02 22:28:36.793332 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.791411    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-dshm\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-dshm\") pod \"02407f93-fd96-49c9-a18c-348651482119\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") "
Nov 02 22:28:36.793332 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.791555    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-home\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-home\") pod \"02407f93-fd96-49c9-a18c-348651482119\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") "
Nov 02 22:28:36.793332 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.791601    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02407f93-fd96-49c9-a18c-348651482119-kube-api-access-k7cr2\" (UniqueName: \"kubernetes.io/projected/02407f93-fd96-49c9-a18c-348651482119-kube-api-access-k7cr2\") pod \"02407f93-fd96-49c9-a18c-348651482119\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") "
Nov 02 22:28:36.793332 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.791659    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-model-cache\") pod \"02407f93-fd96-49c9-a18c-348651482119\" (UID: \"02407f93-fd96-49c9-a18c-348651482119\") "
Nov 02 22:28:36.794254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.793712    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-tmp-dir" pod "02407f93-fd96-49c9-a18c-348651482119" (UID: "02407f93-fd96-49c9-a18c-348651482119"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.794254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.794118    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-home" pod "02407f93-fd96-49c9-a18c-348651482119" (UID: "02407f93-fd96-49c9-a18c-348651482119"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795237    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795267    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e83f7463-2538-40cb-a284-ee2179b6d690-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795281    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795299    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9xlq\" (UniqueName: \"kubernetes.io/projected/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-kube-api-access-m9xlq\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795313    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795327    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tr8pn\" (UniqueName: \"kubernetes.io/projected/e83f7463-2538-40cb-a284-ee2179b6d690-kube-api-access-tr8pn\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795342    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795355    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795405    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795418    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795432    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e83f7463-2538-40cb-a284-ee2179b6d690-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795447    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.795686    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.796965 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.796868    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-dshm" pod "02407f93-fd96-49c9-a18c-348651482119" (UID: "02407f93-fd96-49c9-a18c-348651482119"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.796965 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.796938    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-model-cache" pod "02407f93-fd96-49c9-a18c-348651482119" (UID: "02407f93-fd96-49c9-a18c-348651482119"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:28:36.798449 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.798359    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02407f93-fd96-49c9-a18c-348651482119-kube-api-access-k7cr2" pod "02407f93-fd96-49c9-a18c-348651482119" (UID: "02407f93-fd96-49c9-a18c-348651482119"). InnerVolumeSpecName "kube-api-access-k7cr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:28:36.798550 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.798453    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02407f93-fd96-49c9-a18c-348651482119-tls-certs" pod "02407f93-fd96-49c9-a18c-348651482119" (UID: "02407f93-fd96-49c9-a18c-348651482119"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:28:36.898051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.898011    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7cr2\" (UniqueName: \"kubernetes.io/projected/02407f93-fd96-49c9-a18c-348651482119-kube-api-access-k7cr2\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.898291 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.898272    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.898478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.898460    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02407f93-fd96-49c9-a18c-348651482119-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:36.898602 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:36.898590    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02407f93-fd96-49c9-a18c-348651482119-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:28:37.042938 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.042844    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"]
Nov 02 22:28:37.044495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.044468    5067 cpu_manager.go:414] "containerMap: removing container" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main"
Nov 02 22:28:37.044690 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.044672    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main"
Nov 02 22:28:37.044825 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.044811    5067 cpu_manager.go:414] "containerMap: removing container" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main"
Nov 02 22:28:37.044932 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.044918    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main"
Nov 02 22:28:37.045109 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.045095    5067 cpu_manager.go:414] "containerMap: removing container" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main"
Nov 02 22:28:37.045224 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.045211    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main"
Nov 02 22:28:37.045531 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.045509    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" containerName="main"
Nov 02 22:28:37.045661 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.045648    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main"
Nov 02 22:28:37.045793 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.045780    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="02407f93-fd96-49c9-a18c-348651482119" containerName="main"
Nov 02 22:28:37.045899 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.045885    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ddee653-e8f0-42a6-9c14-3361ccc6dc97" containerName="istio-proxy"
Nov 02 22:28:37.051025 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.051004    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.114978 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.113910    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-db36b079-serving-cert\""
Nov 02 22:28:37.114978 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.114346    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a8dcab-fda2-4e49-9258-c7f701958f32-openshift-service-ca\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.114978 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.114506    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zqb4\" (UniqueName: \"kubernetes.io/projected/71a8dcab-fda2-4e49-9258-c7f701958f32-kube-api-access-5zqb4\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.114978 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.114657    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-metrics-certs\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.114978 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.114855    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-maas-api-tls\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.121193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.121165    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"]
Nov 02 22:28:37.236425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.224720    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zqb4\" (UniqueName: \"kubernetes.io/projected/71a8dcab-fda2-4e49-9258-c7f701958f32-kube-api-access-5zqb4\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.236648 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.236617    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-metrics-certs\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.236830 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.236808    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-maas-api-tls\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.236906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.236885    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a8dcab-fda2-4e49-9258-c7f701958f32-openshift-service-ca\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.238626 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:37.237527    5067 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-serving-cert: secret "maas-api-e2e-ait-db36b079-serving-cert" not found
Nov 02 22:28:37.238626 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:37.237599    5067 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-maas-api-tls podName:71a8dcab-fda2-4e49-9258-c7f701958f32 nodeName:}" failed. No retries permitted until 2026-11-02 22:28:37.737579505 +0000 UTC m=+1390.021389187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-maas-api-tls") pod "maas-api-e2e-ait-db36b079-56967d848d-rbhwq" (UID: "71a8dcab-fda2-4e49-9258-c7f701958f32") : secret "maas-api-e2e-ait-db36b079-serving-cert" not found
Nov 02 22:28:37.238626 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.237796    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a8dcab-fda2-4e49-9258-c7f701958f32-openshift-service-ca\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.279696 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.279659    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-metrics-certs\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.411441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.395779    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zqb4\" (UniqueName: \"kubernetes.io/projected/71a8dcab-fda2-4e49-9258-c7f701958f32-kube-api-access-5zqb4\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.552393 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.540440    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:37.588825 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.588784    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk" event={"ID":"e83f7463-2538-40cb-a284-ee2179b6d690","Type":"ContainerDied","Data":"67d7f0e6da5e5bdda3743e420881bdaff2f9a30b6c576ed21f2b61fc9b5f8d66"}
Nov 02 22:28:37.607434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.605107    5067 scope.go:122] "RemoveContainer" containerID="a025d17782de4732d4cb9150a5a3ff3b2ce46f611de207df641583e86a51c297"
Nov 02 22:28:37.607434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.591560    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"
Nov 02 22:28:37.627983 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.625592    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn" event={"ID":"eaad5c27-f022-46cd-9f1a-a8f3390fe0a5","Type":"ContainerDied","Data":"82e30794bc1aaf76feaee70b31dca9d5eebded49523975510cb643be2da705da"}
Nov 02 22:28:37.627983 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.625919    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"
Nov 02 22:28:37.676969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.675916    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds" event={"ID":"02407f93-fd96-49c9-a18c-348651482119","Type":"ContainerDied","Data":"1b30d230e37ee2ea6be94357bbcd8113ca7b71abc68b713a91ae1997af9be4ea"}
Nov 02 22:28:37.676969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.676011    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"
Nov 02 22:28:37.722347 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:28:37.717900    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:28:37.761507 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.761478    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-maas-api-tls\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.789266 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.789175    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-maas-api-tls\") pod \"maas-api-e2e-ait-db36b079-56967d848d-rbhwq\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:37.832552 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.832519    5067 scope.go:122] "RemoveContainer" containerID="e86f37a930f7ebe0b74347037fbcf5065f5bfd12c2b9bedabc9f7ddd0109714a"
Nov 02 22:28:37.853801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.853093    5067 scope.go:122] "RemoveContainer" containerID="bdcd1a4cbb8ebd728f2ddee03b7942e7ee4a9636527bf14f1fd375719ed3ad6f"
Nov 02 22:28:37.932055 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.931479    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"]
Nov 02 22:28:37.995389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:37.975853    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/simulator-w0-711bd4-kserve-5dbd6ccbb6-5fnrk"]
Nov 02 22:28:38.031785 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:38.031754    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"]
Nov 02 22:28:38.048163 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:38.041438    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/distinct-2-w0-711bd4-kserve-6b99f85f69-x24nn"]
Nov 02 22:28:38.048163 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:38.044666    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:38.206446 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:38.205411    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"]
Nov 02 22:28:38.232572 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:38.232538    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-711bd4/premium-w0-711bd4-kserve-7bb5f5c567-dpwds"]
Nov 02 22:28:39.419390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:39.417077    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"]
Nov 02 22:28:40.105389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.105306    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:40.178422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.177339    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 02 22:28:40.441449 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.388571    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02407f93-fd96-49c9-a18c-348651482119" path="/var/lib/kubelet/pods/02407f93-fd96-49c9-a18c-348651482119/volumes"
Nov 02 22:28:40.475969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.473726    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e83f7463-2538-40cb-a284-ee2179b6d690" path="/var/lib/kubelet/pods/e83f7463-2538-40cb-a284-ee2179b6d690/volumes"
Nov 02 22:28:40.475969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.474472    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eaad5c27-f022-46cd-9f1a-a8f3390fe0a5" path="/var/lib/kubelet/pods/eaad5c27-f022-46cd-9f1a-a8f3390fe0a5/volumes"
Nov 02 22:28:40.475969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.474925    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq" event={"ID":"71a8dcab-fda2-4e49-9258-c7f701958f32","Type":"ContainerStarted","Data":"c2a6c426ff0c6d09a1a59f58d5be407e31a007dd7274dc0c0b0f4721ef3fd8e1"}
Nov 02 22:28:40.475969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.455163    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:40.475969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.432128    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:28:40.545986 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.545942    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:40.700440 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.699473    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:40.700440 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:40.699922    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 22:28:41.487071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:41.487022    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:41.487923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:41.487889    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:28:41.508580 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:41.506926    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:28:41.780168 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:41.780136    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:28:42.193202 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:42.189690    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq" event={"ID":"71a8dcab-fda2-4e49-9258-c7f701958f32","Type":"ContainerStarted","Data":"13471363b308d7bb63cfc1e2b63ec48e6439964df5a7b33268fc5d17ef51ab81"}
Nov 02 22:28:42.193202 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:42.189740    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:42.303168 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:42.303112    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq" podStartSLOduration=4.9109200600000005 podStartE2EDuration="6.303096633s" totalImagesPullingTime="1.392176573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:28:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:28:42.299418299 +0000 UTC m=+1394.583228011" watchObservedRunningTime="2026-11-02 22:28:42.303096633 +0000 UTC m=+1394.586906319"
Nov 02 22:28:45.053388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.052445    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"]
Nov 02 22:28:45.057086 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.057060    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:28:45.110963 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.110923    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"]
Nov 02 22:28:45.126468 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.126422    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f07856b-fc5a-40ab-98b0-540209c4b67d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct\" (UID: \"8f07856b-fc5a-40ab-98b0-540209c4b67d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:28:45.126684 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.126666    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wfvz\" (UniqueName: \"kubernetes.io/projected/8f07856b-fc5a-40ab-98b0-540209c4b67d-kube-api-access-8wfvz\") pod \"payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct\" (UID: \"8f07856b-fc5a-40ab-98b0-540209c4b67d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:28:45.151592 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.151553    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-db36b079\""
Nov 02 22:28:45.151808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.151789    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-db36b079-dockercfg-8gtd6\""
Nov 02 22:28:45.217398 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.214596    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"]
Nov 02 22:28:45.222418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.222345    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:28:45.229527 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.228322    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wfvz\" (UniqueName: \"kubernetes.io/projected/8f07856b-fc5a-40ab-98b0-540209c4b67d-kube-api-access-8wfvz\") pod \"payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct\" (UID: \"8f07856b-fc5a-40ab-98b0-540209c4b67d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:28:45.229527 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.228542    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f07856b-fc5a-40ab-98b0-540209c4b67d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct\" (UID: \"8f07856b-fc5a-40ab-98b0-540209c4b67d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:28:45.229527 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.229249    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f07856b-fc5a-40ab-98b0-540209c4b67d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct\" (UID: \"8f07856b-fc5a-40ab-98b0-540209c4b67d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:28:45.286157 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.286112    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"]
Nov 02 22:28:45.288604 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.288189    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wfvz\" (UniqueName: \"kubernetes.io/projected/8f07856b-fc5a-40ab-98b0-540209c4b67d-kube-api-access-8wfvz\") pod \"payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct\" (UID: \"8f07856b-fc5a-40ab-98b0-540209c4b67d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:28:45.374453 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.374297    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-plugins-config-volume\") pod \"payload-processing-e2e-ait-db36b079-749d6c5485-h67rk\" (UID: \"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0\") " pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:28:45.378286 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.377380    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gvwz\" (UniqueName: \"kubernetes.io/projected/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-kube-api-access-2gvwz\") pod \"payload-processing-e2e-ait-db36b079-749d6c5485-h67rk\" (UID: \"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0\") " pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:28:45.378286 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.378009    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:28:45.488404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.488188    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-plugins-config-volume\") pod \"payload-processing-e2e-ait-db36b079-749d6c5485-h67rk\" (UID: \"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0\") " pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:28:45.488404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.488293    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvwz\" (UniqueName: \"kubernetes.io/projected/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-kube-api-access-2gvwz\") pod \"payload-processing-e2e-ait-db36b079-749d6c5485-h67rk\" (UID: \"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0\") " pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:28:45.504722 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.489622    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-plugins-config-volume\") pod \"payload-processing-e2e-ait-db36b079-749d6c5485-h67rk\" (UID: \"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0\") " pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:28:45.611035 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.611000    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gvwz\" (UniqueName: \"kubernetes.io/projected/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-kube-api-access-2gvwz\") pod \"payload-processing-e2e-ait-db36b079-749d6c5485-h67rk\" (UID: \"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0\") " pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:28:45.843049 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:45.843019    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:28:46.046743 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:46.046698    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"]
Nov 02 22:28:46.051914 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:28:46.051859    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f07856b_fc5a_40ab_98b0_540209c4b67d.slice/crio-c51be7e0f4c6e61176bf6b65d764789af510b5c183b7af4f0851b6bd80b79a98 WatchSource:0}: Error finding container c51be7e0f4c6e61176bf6b65d764789af510b5c183b7af4f0851b6bd80b79a98: Status 404 returned error can't find the container with id c51be7e0f4c6e61176bf6b65d764789af510b5c183b7af4f0851b6bd80b79a98
Nov 02 22:28:46.224434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:46.224329    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct" event={"ID":"8f07856b-fc5a-40ab-98b0-540209c4b67d","Type":"ContainerStarted","Data":"7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5"}
Nov 02 22:28:46.224434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:46.224403    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct" event={"ID":"8f07856b-fc5a-40ab-98b0-540209c4b67d","Type":"ContainerStarted","Data":"c51be7e0f4c6e61176bf6b65d764789af510b5c183b7af4f0851b6bd80b79a98"}
Nov 02 22:28:46.224946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:46.224447    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:28:46.300994 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:46.300963    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"]
Nov 02 22:28:46.303116 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:28:46.303074    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4809ca1_e3c2_4c37_9e6f_988d37fc1aa0.slice/crio-2eeaafba3ce61baf62a41389b09054aac8d022e23e2b805638175232a14bcdeb WatchSource:0}: Error finding container 2eeaafba3ce61baf62a41389b09054aac8d022e23e2b805638175232a14bcdeb: Status 404 returned error can't find the container with id 2eeaafba3ce61baf62a41389b09054aac8d022e23e2b805638175232a14bcdeb
Nov 02 22:28:47.234985 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:47.234945    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk" event={"ID":"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0","Type":"ContainerStarted","Data":"792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629"}
Nov 02 22:28:47.235544 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:47.235022    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:28:47.235544 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:47.235044    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk" event={"ID":"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0","Type":"ContainerStarted","Data":"2eeaafba3ce61baf62a41389b09054aac8d022e23e2b805638175232a14bcdeb"}
Nov 02 22:28:47.353603 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:47.353548    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk" podStartSLOduration=2.353533262 podStartE2EDuration="2.353533262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:28:47.350719269 +0000 UTC m=+1399.634528975" watchObservedRunningTime="2026-11-02 22:28:47.353533262 +0000 UTC m=+1399.637342949"
Nov 02 22:28:47.353943 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:47.353923    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct" podStartSLOduration=2.35391854 podStartE2EDuration="2.35391854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:28:46.337868909 +0000 UTC m=+1398.621678597" watchObservedRunningTime="2026-11-02 22:28:47.35391854 +0000 UTC m=+1399.637728256"
Nov 02 22:28:48.201432 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:48.201402    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:28:57.238669 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:57.238638    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:28:58.246934 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:28:58.246903    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:29:00.724032 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.723999    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"]
Nov 02 22:29:00.727495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.727478    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.786613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.786579    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.786801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.786624    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.786886 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.786860    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pzvv\" (UniqueName: \"kubernetes.io/projected/f7a52edf-7fef-427a-99cd-4add52eaf196-kube-api-access-5pzvv\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.786949 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.786928    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.787010 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.786996    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.787065 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.787024    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a52edf-7fef-427a-99cd-4add52eaf196-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.787065 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.787050    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.787200 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.787185    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.787264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.787227    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.825168 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.825135    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"]
Nov 02 22:29:00.888590 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.888557    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.888590 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.888594    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a52edf-7fef-427a-99cd-4add52eaf196-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.888857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.888611    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.888857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.888650    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.888857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.888704    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.888857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.888797    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.888857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.888831    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.889163 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.889113    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pzvv\" (UniqueName: \"kubernetes.io/projected/f7a52edf-7fef-427a-99cd-4add52eaf196-kube-api-access-5pzvv\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.889222 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.889178    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.889222 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.889196    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.889335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.889295    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.889335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.889318    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.889625 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.889602    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7a52edf-7fef-427a-99cd-4add52eaf196-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.889919 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.889895    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.891688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.891660    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.892106 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.892080    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.947522 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.947489    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7a52edf-7fef-427a-99cd-4add52eaf196-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:00.947681 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:00.947630    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pzvv\" (UniqueName: \"kubernetes.io/projected/f7a52edf-7fef-427a-99cd-4add52eaf196-kube-api-access-5pzvv\") pod \"maas-default-gateway-openshift-default-76f4876667-mxjzx\" (UID: \"f7a52edf-7fef-427a-99cd-4add52eaf196\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:01.040760 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:01.040670    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:01.472129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:01.472094    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"]
Nov 02 22:29:01.472249 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:29:01.472104    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a52edf_7fef_427a_99cd_4add52eaf196.slice/crio-b627ba23293af6c678e0a79fd1ff78dfb3c51a9beab4d43e98dca1679a145387 WatchSource:0}: Error finding container b627ba23293af6c678e0a79fd1ff78dfb3c51a9beab4d43e98dca1679a145387: Status 404 returned error can't find the container with id b627ba23293af6c678e0a79fd1ff78dfb3c51a9beab4d43e98dca1679a145387
Nov 02 22:29:01.475664 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:01.475629    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:01.475767 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:01.475697    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:01.475767 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:01.475726    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:02.384754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:02.384719    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" event={"ID":"f7a52edf-7fef-427a-99cd-4add52eaf196","Type":"ContainerStarted","Data":"c4644309fab2dbd51e098924520dbd172058897e5614971116e0d6e2cebc5821"}
Nov 02 22:29:02.385250 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:02.384782    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" event={"ID":"f7a52edf-7fef-427a-99cd-4add52eaf196","Type":"ContainerStarted","Data":"b627ba23293af6c678e0a79fd1ff78dfb3c51a9beab4d43e98dca1679a145387"}
Nov 02 22:29:02.616632 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:02.616576    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" podStartSLOduration=2.616560731 podStartE2EDuration="2.616560731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:29:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:29:02.566959807 +0000 UTC m=+1414.850769514" watchObservedRunningTime="2026-11-02 22:29:02.616560731 +0000 UTC m=+1414.900370417"
Nov 02 22:29:03.041776 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:03.041743    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:03.047399 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:03.046484    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-mxjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:29:03.047399 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:03.046552    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" podUID="f7a52edf-7fef-427a-99cd-4add52eaf196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 02 22:29:04.052794 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:04.052749    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-mxjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:29:04.053422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:04.053386    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" podUID="f7a52edf-7fef-427a-99cd-4add52eaf196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 02 22:29:05.044389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:05.042761    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-mxjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:29:05.044389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:05.042826    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" podUID="f7a52edf-7fef-427a-99cd-4add52eaf196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 02 22:29:06.064390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:06.055763    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-mxjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:29:06.064390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:06.055828    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" podUID="f7a52edf-7fef-427a-99cd-4add52eaf196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 02 22:29:07.049386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:07.044490    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-mxjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:29:07.049386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:07.044554    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" podUID="f7a52edf-7fef-427a-99cd-4add52eaf196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 02 22:29:07.808423 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:29:07.807655    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:29:08.042403 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:08.041705    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-mxjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:29:08.042403 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:08.041788    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" podUID="f7a52edf-7fef-427a-99cd-4add52eaf196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 02 22:29:09.041076 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:09.041028    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-mxjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 02 22:29:09.041643 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:09.041092    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" podUID="f7a52edf-7fef-427a-99cd-4add52eaf196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 02 22:29:10.628696 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:10.628659    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:10.629577 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:10.629550    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:10.631407 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:10.631379    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx"
Nov 02 22:29:11.439536 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.439497    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"]
Nov 02 22:29:11.443885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.443862    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:29:11.494614 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.494575    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-2e86dfb1-dockercfg-b62qf\""
Nov 02 22:29:11.494815 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.494678    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-2e86dfb1\""
Nov 02 22:29:11.502964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.502935    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"]
Nov 02 22:29:11.527681 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.527645    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvxtt\" (UniqueName: \"kubernetes.io/projected/7f966b0b-b093-439f-ba90-be6684b0f1ce-kube-api-access-nvxtt\") pod \"payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv\" (UID: \"7f966b0b-b093-439f-ba90-be6684b0f1ce\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:29:11.527891 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.527782    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f966b0b-b093-439f-ba90-be6684b0f1ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv\" (UID: \"7f966b0b-b093-439f-ba90-be6684b0f1ce\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:29:11.630049 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.629999    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvxtt\" (UniqueName: \"kubernetes.io/projected/7f966b0b-b093-439f-ba90-be6684b0f1ce-kube-api-access-nvxtt\") pod \"payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv\" (UID: \"7f966b0b-b093-439f-ba90-be6684b0f1ce\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:29:11.630508 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.630141    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f966b0b-b093-439f-ba90-be6684b0f1ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv\" (UID: \"7f966b0b-b093-439f-ba90-be6684b0f1ce\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:29:11.631040 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.631008    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f966b0b-b093-439f-ba90-be6684b0f1ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv\" (UID: \"7f966b0b-b093-439f-ba90-be6684b0f1ce\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:29:11.689966 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.689888    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvxtt\" (UniqueName: \"kubernetes.io/projected/7f966b0b-b093-439f-ba90-be6684b0f1ce-kube-api-access-nvxtt\") pod \"payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv\" (UID: \"7f966b0b-b093-439f-ba90-be6684b0f1ce\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:29:11.763946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:11.763609    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:29:12.535174 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:12.535141    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"]
Nov 02 22:29:12.694022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:12.693985    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv" event={"ID":"7f966b0b-b093-439f-ba90-be6684b0f1ce","Type":"ContainerStarted","Data":"5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e"}
Nov 02 22:29:12.694543 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:12.694059    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv" event={"ID":"7f966b0b-b093-439f-ba90-be6684b0f1ce","Type":"ContainerStarted","Data":"cc785c7829bfc4acab8860e1510125511b26ada9e89b55277f4c21c8430a9e7c"}
Nov 02 22:29:12.694543 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:12.694206    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:29:12.807698 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:12.807592    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv" podStartSLOduration=1.8075778919999999 podStartE2EDuration="1.807577892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:29:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:29:12.803338396 +0000 UTC m=+1425.087148074" watchObservedRunningTime="2026-11-02 22:29:12.807577892 +0000 UTC m=+1425.091387580"
Nov 02 22:29:15.738090 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.738048    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"]
Nov 02 22:29:15.741710 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.741691    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.766023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.765997    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.766159 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.766117    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.766231 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.766171    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.766294 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.766227    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.766395 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.766356    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.766451 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.766427    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.766502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.766463    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsh7t\" (UniqueName: \"kubernetes.io/projected/1d4d7818-f4be-42f8-9db2-28dcbf96091d-kube-api-access-jsh7t\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.766502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.766491    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.766683 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.766666    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.796464 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.793520    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"]
Nov 02 22:29:15.802260 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.802231    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.843512 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.843473    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"]
Nov 02 22:29:15.878456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.878412    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.878613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.878541    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.878613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.878579    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.878698 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.878637    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.878738 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.878700    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.878769 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.878748    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.878823 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.878809    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.878889 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.878876    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.878967 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.878953    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.879017 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.879001    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.879052 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.879038    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72c6a764-cb40-417c-ae95-bde8fd50e29e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.879101 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.879088    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.879151 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.879136    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.879199 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.879185    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6pm4\" (UniqueName: \"kubernetes.io/projected/72c6a764-cb40-417c-ae95-bde8fd50e29e-kube-api-access-b6pm4\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.879235 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.879218    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsh7t\" (UniqueName: \"kubernetes.io/projected/1d4d7818-f4be-42f8-9db2-28dcbf96091d-kube-api-access-jsh7t\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.879270 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.879252    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.879304 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.879277    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.879339 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.879311    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.880177 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.880153    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.880962 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.880935    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.881129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.880952    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.881548 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.881526    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.882010 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.881987    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.886726 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.886701    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.887961 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.887881    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.888914 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.888896    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"]
Nov 02 22:29:15.943754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.943715    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsh7t\" (UniqueName: \"kubernetes.io/projected/1d4d7818-f4be-42f8-9db2-28dcbf96091d-kube-api-access-jsh7t\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.943944 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.943715    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d4d7818-f4be-42f8-9db2-28dcbf96091d-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-4jg6k\" (UID: \"1d4d7818-f4be-42f8-9db2-28dcbf96091d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:15.990427 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.990325    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.990778 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.990753    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72c6a764-cb40-417c-ae95-bde8fd50e29e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.990918 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.990893    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-workload-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.991072 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.991044    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6pm4\" (UniqueName: \"kubernetes.io/projected/72c6a764-cb40-417c-ae95-bde8fd50e29e-kube-api-access-b6pm4\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.991192 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.991091    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.991192 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.991115    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.991192 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.991164    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.991312 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.991253    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.991312 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.991305    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.991457 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.991440    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.991548 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.991527    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72c6a764-cb40-417c-ae95-bde8fd50e29e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.991861 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.991844    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-data\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.992085 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.992069    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-workload-certs\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.992626 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.992598    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-credential-socket\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.995345 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.995320    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-envoy\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:15.995688 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:15.995670    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:16.050379 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.050323    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6pm4\" (UniqueName: \"kubernetes.io/projected/72c6a764-cb40-417c-ae95-bde8fd50e29e-kube-api-access-b6pm4\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:16.050872 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.050833    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72c6a764-cb40-417c-ae95-bde8fd50e29e-istio-token\") pod \"maas-default-gateway-openshift-default-76f4876667-m5kv8\" (UID: \"72c6a764-cb40-417c-ae95-bde8fd50e29e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:16.059438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.059407    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:16.120219 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.117483    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:16.488746 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.488712    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"]
Nov 02 22:29:16.489455 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:29:16.489424    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4d7818_f4be_42f8_9db2_28dcbf96091d.slice/crio-96e7d0ed870a664f059f24f26061e6196cd29ecd28457905dbd9df2903358230 WatchSource:0}: Error finding container 96e7d0ed870a664f059f24f26061e6196cd29ecd28457905dbd9df2903358230: Status 404 returned error can't find the container with id 96e7d0ed870a664f059f24f26061e6196cd29ecd28457905dbd9df2903358230
Nov 02 22:29:16.492323 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.492286    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:16.492468 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.492380    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:16.492468 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.492434    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:16.542826 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.542800    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"]
Nov 02 22:29:16.544942 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:29:16.544733    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72c6a764_cb40_417c_ae95_bde8fd50e29e.slice/crio-5868bc1a782bd3517f42fe55737b8e99ae2eba00a1f348a10694c9567bb4354d WatchSource:0}: Error finding container 5868bc1a782bd3517f42fe55737b8e99ae2eba00a1f348a10694c9567bb4354d: Status 404 returned error can't find the container with id 5868bc1a782bd3517f42fe55737b8e99ae2eba00a1f348a10694c9567bb4354d
Nov 02 22:29:16.547779 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.547716    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:16.547881 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.547797    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:16.547881 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.547836    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:16.730023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.728024    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" event={"ID":"1d4d7818-f4be-42f8-9db2-28dcbf96091d","Type":"ContainerStarted","Data":"a94815e9ea178c9a83686d1febc1e0d0de51f0a20773c4452784e1ebc3025724"}
Nov 02 22:29:16.730023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.728080    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" event={"ID":"1d4d7818-f4be-42f8-9db2-28dcbf96091d","Type":"ContainerStarted","Data":"96e7d0ed870a664f059f24f26061e6196cd29ecd28457905dbd9df2903358230"}
Nov 02 22:29:16.730023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.729937    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" event={"ID":"72c6a764-cb40-417c-ae95-bde8fd50e29e","Type":"ContainerStarted","Data":"54bc01cb8d24a79c2821a7000bd172981a452909db0e2f39a888c1ceb71241cd"}
Nov 02 22:29:16.730023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.729993    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" event={"ID":"72c6a764-cb40-417c-ae95-bde8fd50e29e","Type":"ContainerStarted","Data":"5868bc1a782bd3517f42fe55737b8e99ae2eba00a1f348a10694c9567bb4354d"}
Nov 02 22:29:16.905818 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:16.905693    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podStartSLOduration=1.90567416 podStartE2EDuration="1.90567416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:29:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:29:16.903919259 +0000 UTC m=+1429.187728985" watchObservedRunningTime="2026-11-02 22:29:16.90567416 +0000 UTC m=+1429.189483847"
Nov 02 22:29:17.025950 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:17.025891    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" podStartSLOduration=2.025877597 podStartE2EDuration="2.025877597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:29:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:29:17.022740906 +0000 UTC m=+1429.306550613" watchObservedRunningTime="2026-11-02 22:29:17.025877597 +0000 UTC m=+1429.309687315"
Nov 02 22:29:17.060023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:17.059986    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:17.061406 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:17.061348    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:29:17.061557 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:17.061423    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 22:29:17.118033 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:17.118003    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:17.119452 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:17.119424    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-m5kv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 22:29:17.119584 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:17.119477    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" podUID="72c6a764-cb40-417c-ae95-bde8fd50e29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 22:29:18.060769 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:18.060715    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:29:18.061458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:18.061413    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 22:29:18.118333 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:18.118303    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-m5kv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 22:29:18.118595 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:18.118575    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" podUID="72c6a764-cb40-417c-ae95-bde8fd50e29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 22:29:19.069396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:19.066465    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:29:19.069396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:19.066530    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 22:29:19.117958 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:19.117929    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-m5kv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 22:29:19.118182 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:19.118156    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" podUID="72c6a764-cb40-417c-ae95-bde8fd50e29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 22:29:20.065391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:20.062715    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:29:20.065391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:20.062778    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 22:29:20.118713 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:20.118680    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-m5kv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 22:29:20.119177 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:20.119152    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" podUID="72c6a764-cb40-417c-ae95-bde8fd50e29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 22:29:21.067392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:21.066794    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:29:21.067392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:21.066860    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 22:29:21.119480 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:21.118690    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-m5kv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 22:29:21.119480 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:21.118751    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" podUID="72c6a764-cb40-417c-ae95-bde8fd50e29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 22:29:22.063871 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:22.063830    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:29:22.064161 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:22.064132    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 22:29:22.118732 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:22.118700    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-m5kv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 22:29:22.118910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:22.118748    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" podUID="72c6a764-cb40-417c-ae95-bde8fd50e29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 22:29:23.064559 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:23.064519    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:29:23.065150 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:23.065113    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 22:29:23.118022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:23.117987    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-m5kv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 22:29:23.118203 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:23.118042    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" podUID="72c6a764-cb40-417c-ae95-bde8fd50e29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 22:29:23.706853 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:23.706819    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:29:24.060459 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:24.060421    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 22:29:24.060672 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:24.060487    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 22:29:24.118293 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:24.118259    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-m5kv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 22:29:24.118711 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:24.118307    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8" podUID="72c6a764-cb40-417c-ae95-bde8fd50e29e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 22:29:25.995401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:25.994107    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:25.995401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:25.994790    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:25.996556 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:25.996524    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-m5kv8"
Nov 02 22:29:26.100534 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:26.100491    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:29:26.100740 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:26.100538    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:29:26.206029 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:26.205932    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:26.206807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:26.206778    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:26.207608 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:26.207564    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k"
Nov 02 22:29:37.583740 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:29:37.583670    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:29:39.266297 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:39.266264    5067 generic.go:356] "Generic (PLEG): container finished" podID="8881e503-831c-4519-b1b1-75312e378a79" containerID="e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047" exitCode=0
Nov 02 22:29:39.266781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:39.266341    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" event={"ID":"8881e503-831c-4519-b1b1-75312e378a79","Type":"ContainerDied","Data":"e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047"}
Nov 02 22:29:39.266926 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:39.266909    5067 scope.go:122] "RemoveContainer" containerID="e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047"
Nov 02 22:29:39.268643 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:39.268615    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:39.268750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:39.268700    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:39.268833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:39.268747    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:29:40.275680 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:40.275648    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" event={"ID":"8881e503-831c-4519-b1b1-75312e378a79","Type":"ContainerStarted","Data":"fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4"}
Nov 02 22:29:40.385333 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:40.385301    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:29:40.386186 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:40.386151    5067 patch_prober.go:29] interesting pod/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:29:40.386293 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:40.386199    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 02 22:29:41.384751 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:41.384720    5067 patch_prober.go:29] interesting pod/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:29:41.385162 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:41.384772    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 02 22:29:42.281564 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:42.281526    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:29:42.384944 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:42.384912    5067 patch_prober.go:29] interesting pod/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:29:42.385349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:42.384978    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 02 22:29:43.384739 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:43.384704    5067 patch_prober.go:29] interesting pod/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:29:43.385035 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:43.385008    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 02 22:29:44.384946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:44.384905    5067 patch_prober.go:29] interesting pod/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:29:44.385704 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:44.385666    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 02 22:29:45.385051 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:45.385001    5067 patch_prober.go:29] interesting pod/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 02 22:29:45.385247 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:45.385104    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 02 22:29:46.389354 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:46.389324    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:29:46.390573 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:46.390549    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:29:50.557127 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:50.557088    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"]
Nov 02 22:29:50.660972 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:50.660931    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"]
Nov 02 22:29:55.558316 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:55.558248    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk" podUID="d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0" containerName="payload-processing" containerID="cri-o://792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629" gracePeriod=25
Nov 02 22:29:55.662152 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:55.662090    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct" podUID="8f07856b-fc5a-40ab-98b0-540209c4b67d" containerName="payload-pre-processing" containerID="cri-o://7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5" gracePeriod=25
Nov 02 22:29:55.876059 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:55.876024    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:29:55.970431 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:55.970394    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-plugins-config-volume\") pod \"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0\" (UID: \"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0\") "
Nov 02 22:29:55.970566 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:55.970550    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-kube-api-access-2gvwz\" (UniqueName: \"kubernetes.io/projected/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-kube-api-access-2gvwz\") pod \"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0\" (UID: \"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0\") "
Nov 02 22:29:55.970850 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:55.970819    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-plugins-config-volume" pod "d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0" (UID: "d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:29:55.971536 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:55.971508    5067 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-plugins-config-volume\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:29:55.972914 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:55.972887    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-kube-api-access-2gvwz" pod "d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0" (UID: "d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0"). InnerVolumeSpecName "kube-api-access-2gvwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:29:55.988079 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:55.988054    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:29:56.073053 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.073003    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f07856b-fc5a-40ab-98b0-540209c4b67d-kube-api-access-8wfvz\" (UniqueName: \"kubernetes.io/projected/8f07856b-fc5a-40ab-98b0-540209c4b67d-kube-api-access-8wfvz\") pod \"8f07856b-fc5a-40ab-98b0-540209c4b67d\" (UID: \"8f07856b-fc5a-40ab-98b0-540209c4b67d\") "
Nov 02 22:29:56.073288 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.073269    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f07856b-fc5a-40ab-98b0-540209c4b67d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f07856b-fc5a-40ab-98b0-540209c4b67d-plugins-config-volume\") pod \"8f07856b-fc5a-40ab-98b0-540209c4b67d\" (UID: \"8f07856b-fc5a-40ab-98b0-540209c4b67d\") "
Nov 02 22:29:56.073666 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.073643    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f07856b-fc5a-40ab-98b0-540209c4b67d-plugins-config-volume" pod "8f07856b-fc5a-40ab-98b0-540209c4b67d" (UID: "8f07856b-fc5a-40ab-98b0-540209c4b67d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:29:56.073945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.073925    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gvwz\" (UniqueName: \"kubernetes.io/projected/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0-kube-api-access-2gvwz\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:29:56.074018 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.073950    5067 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f07856b-fc5a-40ab-98b0-540209c4b67d-plugins-config-volume\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:29:56.075978 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.075914    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f07856b-fc5a-40ab-98b0-540209c4b67d-kube-api-access-8wfvz" pod "8f07856b-fc5a-40ab-98b0-540209c4b67d" (UID: "8f07856b-fc5a-40ab-98b0-540209c4b67d"). InnerVolumeSpecName "kube-api-access-8wfvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:29:56.175735 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.175679    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wfvz\" (UniqueName: \"kubernetes.io/projected/8f07856b-fc5a-40ab-98b0-540209c4b67d-kube-api-access-8wfvz\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:29:56.412212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.412124    5067 generic.go:356] "Generic (PLEG): container finished" podID="8f07856b-fc5a-40ab-98b0-540209c4b67d" containerID="7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5" exitCode=0
Nov 02 22:29:56.412212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.412169    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"
Nov 02 22:29:56.412212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.412201    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct" event={"ID":"8f07856b-fc5a-40ab-98b0-540209c4b67d","Type":"ContainerDied","Data":"7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5"}
Nov 02 22:29:56.412576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.412251    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct" event={"ID":"8f07856b-fc5a-40ab-98b0-540209c4b67d","Type":"ContainerDied","Data":"c51be7e0f4c6e61176bf6b65d764789af510b5c183b7af4f0851b6bd80b79a98"}
Nov 02 22:29:56.412576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.412273    5067 scope.go:122] "RemoveContainer" containerID="7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5"
Nov 02 22:29:56.415165 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.415138    5067 generic.go:356] "Generic (PLEG): container finished" podID="d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0" containerID="792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629" exitCode=0
Nov 02 22:29:56.415290 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.415180    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"
Nov 02 22:29:56.415290 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.415203    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk" event={"ID":"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0","Type":"ContainerDied","Data":"792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629"}
Nov 02 22:29:56.415290 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.415258    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk" event={"ID":"d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0","Type":"ContainerDied","Data":"2eeaafba3ce61baf62a41389b09054aac8d022e23e2b805638175232a14bcdeb"}
Nov 02 22:29:56.423484 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.423457    5067 scope.go:122] "RemoveContainer" containerID="7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5"
Nov 02 22:29:56.423800 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:29:56.423777    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5\": container with ID starting with 7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5 not found: ID does not exist" containerID="7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5"
Nov 02 22:29:56.423896 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.423807    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5"} err="failed to get container status \"7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5\": rpc error: code = NotFound desc = could not find container \"7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5\": container with ID starting with 7b62263fe8279da80d8560f2e0403b0ad02ac4d99d089a5171995173bd9f36c5 not found: ID does not exist"
Nov 02 22:29:56.423896 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.423823    5067 scope.go:122] "RemoveContainer" containerID="792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629"
Nov 02 22:29:56.433243 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.433226    5067 scope.go:122] "RemoveContainer" containerID="792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629"
Nov 02 22:29:56.433555 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:29:56.433529    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629\": container with ID starting with 792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629 not found: ID does not exist" containerID="792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629"
Nov 02 22:29:56.433644 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.433564    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629"} err="failed to get container status \"792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629\": rpc error: code = NotFound desc = could not find container \"792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629\": container with ID starting with 792ef46a5ae62c2da803eae02ef7c97982a5a1e08b8f90b841c1225b7d792629 not found: ID does not exist"
Nov 02 22:29:56.571184 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.571145    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"]
Nov 02 22:29:56.577316 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.577283    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-db36b079-749d6c5485-h67rk"]
Nov 02 22:29:56.677693 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.677574    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"]
Nov 02 22:29:56.683656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:56.683625    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-db36b079-f865fbb47-zfjct"]
Nov 02 22:29:57.316958 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.316918    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"]
Nov 02 22:29:57.317205 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.317175    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5" podUID="c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1" containerName="maas-api" containerID="cri-o://f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3" gracePeriod=30
Nov 02 22:29:57.799299 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.799005    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:29:57.911933 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.911894    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f07856b-fc5a-40ab-98b0-540209c4b67d" path="/var/lib/kubelet/pods/8f07856b-fc5a-40ab-98b0-540209c4b67d/volumes"
Nov 02 22:29:57.913103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.913081    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0" path="/var/lib/kubelet/pods/d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0/volumes"
Nov 02 22:29:57.925416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.924036    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-openshift-service-ca\") pod \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") "
Nov 02 22:29:57.925416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.924147    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-kube-api-access-p7tqd\" (UniqueName: \"kubernetes.io/projected/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-kube-api-access-p7tqd\") pod \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") "
Nov 02 22:29:57.925416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.924280    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-maas-api-tls\") pod \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") "
Nov 02 22:29:57.925416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.924339    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-metrics-certs\") pod \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\" (UID: \"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1\") "
Nov 02 22:29:57.927891 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.927010    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-openshift-service-ca" pod "c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1" (UID: "c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:29:57.947679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.946570    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-kube-api-access-p7tqd" pod "c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1" (UID: "c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1"). InnerVolumeSpecName "kube-api-access-p7tqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:29:57.967623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.967023    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-metrics-certs" pod "c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1" (UID: "c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:29:57.967623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:57.967464    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-maas-api-tls" pod "c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1" (UID: "c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:29:58.049450 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.044173    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-maas-api-tls\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:29:58.049450 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.044208    5067 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-metrics-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:29:58.049450 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.044224    5067 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-openshift-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:29:58.049450 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.044240    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7tqd\" (UniqueName: \"kubernetes.io/projected/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1-kube-api-access-p7tqd\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:29:58.463948 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.463856    5067 generic.go:356] "Generic (PLEG): container finished" podID="c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1" containerID="f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3" exitCode=0
Nov 02 22:29:58.464238 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.464218    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5" event={"ID":"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1","Type":"ContainerDied","Data":"f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3"}
Nov 02 22:29:58.464380 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.464349    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5" event={"ID":"c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1","Type":"ContainerDied","Data":"779a6127e0b374c89da813ae7be1f92a2548d5f4b948dfdc57adc5e9a8958f32"}
Nov 02 22:29:58.464505 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.464493    5067 scope.go:122] "RemoveContainer" containerID="f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3"
Nov 02 22:29:58.464738 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.464722    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"
Nov 02 22:29:58.493658 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.493630    5067 scope.go:122] "RemoveContainer" containerID="f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3"
Nov 02 22:29:58.494018 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:29:58.493983    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3\": container with ID starting with f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3 not found: ID does not exist" containerID="f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3"
Nov 02 22:29:58.494106 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.494031    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3"} err="failed to get container status \"f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3\": rpc error: code = NotFound desc = could not find container \"f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3\": container with ID starting with f13432133d901c7c7b1492b2e983c8f75620f5a0697b59cf288257759b113ce3 not found: ID does not exist"
Nov 02 22:29:58.568903 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:29:58.568867    5067 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23b9c5e_72b9_4b4e_9cfd_21f7a01796b1.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23b9c5e_72b9_4b4e_9cfd_21f7a01796b1.slice/crio-779a6127e0b374c89da813ae7be1f92a2548d5f4b948dfdc57adc5e9a8958f32\": RecentStats: unable to find data in memory cache]"
Nov 02 22:29:58.626631 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.626593    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"]
Nov 02 22:29:58.671023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:58.670989    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-711bd4-84d5499c78-xbnq5"]
Nov 02 22:29:59.858985 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:29:59.858953    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1" path="/var/lib/kubelet/pods/c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1/volumes"
Nov 02 22:30:00.168522 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.168445    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"]
Nov 02 22:30:00.169106 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.169092    5067 cpu_manager.go:414] "containerMap: removing container" podUID="c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1" containerName="maas-api"
Nov 02 22:30:00.169151 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.169109    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1" containerName="maas-api"
Nov 02 22:30:00.169151 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.169128    5067 cpu_manager.go:414] "containerMap: removing container" podUID="8f07856b-fc5a-40ab-98b0-540209c4b67d" containerName="payload-pre-processing"
Nov 02 22:30:00.169151 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.169133    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f07856b-fc5a-40ab-98b0-540209c4b67d" containerName="payload-pre-processing"
Nov 02 22:30:00.169151 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.169151    5067 cpu_manager.go:414] "containerMap: removing container" podUID="d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0" containerName="payload-processing"
Nov 02 22:30:00.169268 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.169156    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0" containerName="payload-processing"
Nov 02 22:30:00.169304 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.169269    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="c23b9c5e-72b9-4b4e-9cfd-21f7a01796b1" containerName="maas-api"
Nov 02 22:30:00.169304 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.169278    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f07856b-fc5a-40ab-98b0-540209c4b67d" containerName="payload-pre-processing"
Nov 02 22:30:00.169304 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.169286    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4809ca1-e3c2-4c37-9e6f-988d37fc1aa0" containerName="payload-processing"
Nov 02 22:30:00.172877 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.172857    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"
Nov 02 22:30:00.183975 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.183949    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"]
Nov 02 22:30:00.187373 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.187345    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"
Nov 02 22:30:00.187463 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.187414    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"]
Nov 02 22:30:00.190721 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.190701    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"
Nov 02 22:30:00.226119 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.226087    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"]
Nov 02 22:30:00.233006 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.232965    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cndr2\""
Nov 02 22:30:00.271915 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.271870    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"]
Nov 02 22:30:00.276015 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.275981    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"]
Nov 02 22:30:00.301557 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.301522    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp\" (UID: \"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"
Nov 02 22:30:00.301703 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.301680    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b41ad331-309b-4899-98f6-5b313e554e0b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v\" (UID: \"b41ad331-309b-4899-98f6-5b313e554e0b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"
Nov 02 22:30:00.301756 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.301741    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fngx\" (UniqueName: \"kubernetes.io/projected/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-kube-api-access-5fngx\") pod \"maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp\" (UID: \"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"
Nov 02 22:30:00.301889 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.301865    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmng7\" (UniqueName: \"kubernetes.io/projected/b41ad331-309b-4899-98f6-5b313e554e0b-kube-api-access-xmng7\") pod \"maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v\" (UID: \"b41ad331-309b-4899-98f6-5b313e554e0b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"
Nov 02 22:30:00.302036 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.302018    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v\" (UID: \"b9bed3dd-9b4d-4b17-8c58-4edb7916762b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"
Nov 02 22:30:00.302078 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.302062    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bs92\" (UniqueName: \"kubernetes.io/projected/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-kube-api-access-6bs92\") pod \"maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v\" (UID: \"b9bed3dd-9b4d-4b17-8c58-4edb7916762b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"
Nov 02 22:30:00.404069 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.404031    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b41ad331-309b-4899-98f6-5b313e554e0b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v\" (UID: \"b41ad331-309b-4899-98f6-5b313e554e0b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"
Nov 02 22:30:00.404240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.404175    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fngx\" (UniqueName: \"kubernetes.io/projected/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-kube-api-access-5fngx\") pod \"maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp\" (UID: \"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"
Nov 02 22:30:00.404240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.404210    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmng7\" (UniqueName: \"kubernetes.io/projected/b41ad331-309b-4899-98f6-5b313e554e0b-kube-api-access-xmng7\") pod \"maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v\" (UID: \"b41ad331-309b-4899-98f6-5b313e554e0b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"
Nov 02 22:30:00.404341 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.404316    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v\" (UID: \"b9bed3dd-9b4d-4b17-8c58-4edb7916762b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"
Nov 02 22:30:00.404477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.404399    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bs92\" (UniqueName: \"kubernetes.io/projected/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-kube-api-access-6bs92\") pod \"maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v\" (UID: \"b9bed3dd-9b4d-4b17-8c58-4edb7916762b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"
Nov 02 22:30:00.404533 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.404521    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp\" (UID: \"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"
Nov 02 22:30:00.404895 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.404870    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b41ad331-309b-4899-98f6-5b313e554e0b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v\" (UID: \"b41ad331-309b-4899-98f6-5b313e554e0b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"
Nov 02 22:30:00.404979 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.404962    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v\" (UID: \"b9bed3dd-9b4d-4b17-8c58-4edb7916762b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"
Nov 02 22:30:00.405161 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.405145    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp\" (UID: \"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"
Nov 02 22:30:00.461779 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.461706    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmng7\" (UniqueName: \"kubernetes.io/projected/b41ad331-309b-4899-98f6-5b313e554e0b-kube-api-access-xmng7\") pod \"maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v\" (UID: \"b41ad331-309b-4899-98f6-5b313e554e0b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"
Nov 02 22:30:00.462228 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.462203    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bs92\" (UniqueName: \"kubernetes.io/projected/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-kube-api-access-6bs92\") pod \"maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v\" (UID: \"b9bed3dd-9b4d-4b17-8c58-4edb7916762b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"
Nov 02 22:30:00.462534 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.462510    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fngx\" (UniqueName: \"kubernetes.io/projected/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-kube-api-access-5fngx\") pod \"maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp\" (UID: \"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"
Nov 02 22:30:00.486348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.486323    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"
Nov 02 22:30:00.502142 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.502117    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"
Nov 02 22:30:00.508592 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.508564    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"
Nov 02 22:30:00.876907 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.876879    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"]
Nov 02 22:30:00.877635 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:30:00.877612    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41ad331_309b_4899_98f6_5b313e554e0b.slice/crio-be9d1264715c3352d8204908caace921567fe4d28ecaa801b2732792e9640ddd WatchSource:0}: Error finding container be9d1264715c3352d8204908caace921567fe4d28ecaa801b2732792e9640ddd: Status 404 returned error can't find the container with id be9d1264715c3352d8204908caace921567fe4d28ecaa801b2732792e9640ddd
Nov 02 22:30:00.924468 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.924441    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"]
Nov 02 22:30:00.925559 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:30:00.925530    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5290c2f6_d0f8_4a67_aa29_f2b3fb63cda4.slice/crio-0bb465a6d65635f6c86cc7b46b0085a611466b9f620a6bcfe7cec089aa8cdb7c WatchSource:0}: Error finding container 0bb465a6d65635f6c86cc7b46b0085a611466b9f620a6bcfe7cec089aa8cdb7c: Status 404 returned error can't find the container with id 0bb465a6d65635f6c86cc7b46b0085a611466b9f620a6bcfe7cec089aa8cdb7c
Nov 02 22:30:00.973085 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:00.973061    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"]
Nov 02 22:30:00.974883 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:30:00.974859    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9bed3dd_9b4d_4b17_8c58_4edb7916762b.slice/crio-bfed4ff1f3fe7bc54f756bd57b70a1f6935b16a21a71e49155070ecd74f07bc6 WatchSource:0}: Error finding container bfed4ff1f3fe7bc54f756bd57b70a1f6935b16a21a71e49155070ecd74f07bc6: Status 404 returned error can't find the container with id bfed4ff1f3fe7bc54f756bd57b70a1f6935b16a21a71e49155070ecd74f07bc6
Nov 02 22:30:01.545029 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:01.544982    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" event={"ID":"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4","Type":"ContainerStarted","Data":"0bb465a6d65635f6c86cc7b46b0085a611466b9f620a6bcfe7cec089aa8cdb7c"}
Nov 02 22:30:01.546392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:01.546350    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v" event={"ID":"b9bed3dd-9b4d-4b17-8c58-4edb7916762b","Type":"ContainerStarted","Data":"bfed4ff1f3fe7bc54f756bd57b70a1f6935b16a21a71e49155070ecd74f07bc6"}
Nov 02 22:30:01.547647 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:01.547619    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" event={"ID":"b41ad331-309b-4899-98f6-5b313e554e0b","Type":"ContainerStarted","Data":"be9d1264715c3352d8204908caace921567fe4d28ecaa801b2732792e9640ddd"}
Nov 02 22:30:06.591192 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:06.591154    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" event={"ID":"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4","Type":"ContainerStarted","Data":"5f94064c06220316375867d40190e63c0c3697ca481c5e0a77ab958082c9d546"}
Nov 02 22:30:06.593253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:06.593223    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" event={"ID":"b41ad331-309b-4899-98f6-5b313e554e0b","Type":"ContainerStarted","Data":"2bbbfd8bf8a0adaa3dba515d5b49cbc55fceab219efff7e98d88ea3127215260"}
Nov 02 22:30:06.702028 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:06.701922    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" podStartSLOduration=1.8042714370000001 podStartE2EDuration="6.701906263s" totalImagesPullingTime="4.897634826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:30:06.699479804 +0000 UTC m=+1478.983289493" watchObservedRunningTime="2026-11-02 22:30:06.701906263 +0000 UTC m=+1478.985715951"
Nov 02 22:30:06.807981 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:06.807915    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" podStartSLOduration=1.93660791 podStartE2EDuration="6.807894056s" totalImagesPullingTime="4.871286146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:30:06.807549859 +0000 UTC m=+1479.091359559" watchObservedRunningTime="2026-11-02 22:30:06.807894056 +0000 UTC m=+1479.091703747"
Nov 02 22:30:06.980192 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:06.980148    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"]
Nov 02 22:30:06.980594 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:06.980413    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" containerName="istio-proxy" containerID="cri-o://b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89" gracePeriod=30
Nov 02 22:30:06.986946 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:06.986920    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"]
Nov 02 22:30:06.987169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:06.987143    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerName="istio-proxy" containerID="cri-o://1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798" gracePeriod=30
Nov 02 22:30:07.008962 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.008920    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"]
Nov 02 22:30:07.009180 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.009155    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy" containerID="cri-o://8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052" gracePeriod=30
Nov 02 22:30:07.590578 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:07.590531    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:30:07.604032 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.603997    5067 generic.go:356] "Generic (PLEG): container finished" podID="b41ad331-309b-4899-98f6-5b313e554e0b" containerID="2bbbfd8bf8a0adaa3dba515d5b49cbc55fceab219efff7e98d88ea3127215260" exitCode=60
Nov 02 22:30:07.604486 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.604073    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" event={"ID":"b41ad331-309b-4899-98f6-5b313e554e0b","Type":"ContainerDied","Data":"2bbbfd8bf8a0adaa3dba515d5b49cbc55fceab219efff7e98d88ea3127215260"}
Nov 02 22:30:07.604567 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.604512    5067 scope.go:122] "RemoveContainer" containerID="2bbbfd8bf8a0adaa3dba515d5b49cbc55fceab219efff7e98d88ea3127215260"
Nov 02 22:30:07.606118 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.606090    5067 generic.go:356] "Generic (PLEG): container finished" podID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerID="5f94064c06220316375867d40190e63c0c3697ca481c5e0a77ab958082c9d546" exitCode=60
Nov 02 22:30:07.606255 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.606197    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" event={"ID":"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4","Type":"ContainerDied","Data":"5f94064c06220316375867d40190e63c0c3697ca481c5e0a77ab958082c9d546"}
Nov 02 22:30:07.606560 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.606537    5067 scope.go:122] "RemoveContainer" containerID="5f94064c06220316375867d40190e63c0c3697ca481c5e0a77ab958082c9d546"
Nov 02 22:30:07.608275 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.608251    5067 generic.go:356] "Generic (PLEG): container finished" podID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerID="706a5d782ebbc84873c13aa78c86d1861fd1607a074a6fd55f7cbf54d121ea4b" exitCode=60
Nov 02 22:30:07.608733 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.608355    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v" event={"ID":"b9bed3dd-9b4d-4b17-8c58-4edb7916762b","Type":"ContainerDied","Data":"706a5d782ebbc84873c13aa78c86d1861fd1607a074a6fd55f7cbf54d121ea4b"}
Nov 02 22:30:07.608733 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:07.608616    5067 scope.go:122] "RemoveContainer" containerID="706a5d782ebbc84873c13aa78c86d1861fd1607a074a6fd55f7cbf54d121ea4b"
Nov 02 22:30:08.617440 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:08.617400    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" event={"ID":"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4","Type":"ContainerStarted","Data":"9b94cc09b6651333278e7fb9a01b805008392a9cb663a3070acb7742d248bab3"}
Nov 02 22:30:08.619502 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:08.619471    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v" event={"ID":"b9bed3dd-9b4d-4b17-8c58-4edb7916762b","Type":"ContainerStarted","Data":"e18ef49b68b00f381ccf0a720bf87d8851f7b50c96caa15d4690c1c7d405e239"}
Nov 02 22:30:08.621418 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:08.621397    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" event={"ID":"b41ad331-309b-4899-98f6-5b313e554e0b","Type":"ContainerStarted","Data":"78f2fe3fb397a2096f084a68738fb67bb1b60221acbbcd22e1283d4d4cefa0a1"}
Nov 02 22:30:08.945111 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:08.945061    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v" podStartSLOduration=3.55443898 podStartE2EDuration="8.945037935s" totalImagesPullingTime="5.390598955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:30:08.944007266 +0000 UTC m=+1481.227816953" watchObservedRunningTime="2026-11-02 22:30:08.945037935 +0000 UTC m=+1481.228847622"
Nov 02 22:30:09.631750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.631710    5067 generic.go:356] "Generic (PLEG): container finished" podID="b41ad331-309b-4899-98f6-5b313e554e0b" containerID="78f2fe3fb397a2096f084a68738fb67bb1b60221acbbcd22e1283d4d4cefa0a1" exitCode=60
Nov 02 22:30:09.632237 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.631761    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" event={"ID":"b41ad331-309b-4899-98f6-5b313e554e0b","Type":"ContainerDied","Data":"78f2fe3fb397a2096f084a68738fb67bb1b60221acbbcd22e1283d4d4cefa0a1"}
Nov 02 22:30:09.632237 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.631838    5067 scope.go:122] "RemoveContainer" containerID="2bbbfd8bf8a0adaa3dba515d5b49cbc55fceab219efff7e98d88ea3127215260"
Nov 02 22:30:09.632405 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.632245    5067 scope.go:122] "RemoveContainer" containerID="78f2fe3fb397a2096f084a68738fb67bb1b60221acbbcd22e1283d4d4cefa0a1"
Nov 02 22:30:09.632712 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:09.632689    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v_odh-ai-gateway-infra(b41ad331-309b-4899-98f6-5b313e554e0b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" podUID="b41ad331-309b-4899-98f6-5b313e554e0b"
Nov 02 22:30:09.633957 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.633934    5067 generic.go:356] "Generic (PLEG): container finished" podID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerID="9b94cc09b6651333278e7fb9a01b805008392a9cb663a3070acb7742d248bab3" exitCode=60
Nov 02 22:30:09.634076 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.634025    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" event={"ID":"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4","Type":"ContainerDied","Data":"9b94cc09b6651333278e7fb9a01b805008392a9cb663a3070acb7742d248bab3"}
Nov 02 22:30:09.634409 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.634348    5067 scope.go:122] "RemoveContainer" containerID="9b94cc09b6651333278e7fb9a01b805008392a9cb663a3070acb7742d248bab3"
Nov 02 22:30:09.634737 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:09.634715    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp_odh-ai-gateway-infra(5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4"
Nov 02 22:30:09.635976 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.635955    5067 generic.go:356] "Generic (PLEG): container finished" podID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerID="e18ef49b68b00f381ccf0a720bf87d8851f7b50c96caa15d4690c1c7d405e239" exitCode=60
Nov 02 22:30:09.636075 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.636009    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v" event={"ID":"b9bed3dd-9b4d-4b17-8c58-4edb7916762b","Type":"ContainerDied","Data":"e18ef49b68b00f381ccf0a720bf87d8851f7b50c96caa15d4690c1c7d405e239"}
Nov 02 22:30:09.636301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.636286    5067 scope.go:122] "RemoveContainer" containerID="e18ef49b68b00f381ccf0a720bf87d8851f7b50c96caa15d4690c1c7d405e239"
Nov 02 22:30:09.636579 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:09.636562    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v_odh-ai-gateway-infra(b9bed3dd-9b4d-4b17-8c58-4edb7916762b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b"
Nov 02 22:30:09.644714 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.644692    5067 scope.go:122] "RemoveContainer" containerID="5f94064c06220316375867d40190e63c0c3697ca481c5e0a77ab958082c9d546"
Nov 02 22:30:09.657428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:09.657410    5067 scope.go:122] "RemoveContainer" containerID="706a5d782ebbc84873c13aa78c86d1861fd1607a074a6fd55f7cbf54d121ea4b"
Nov 02 22:30:10.644934 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:10.644898    5067 scope.go:122] "RemoveContainer" containerID="e18ef49b68b00f381ccf0a720bf87d8851f7b50c96caa15d4690c1c7d405e239"
Nov 02 22:30:10.645354 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:10.645293    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v_odh-ai-gateway-infra(b9bed3dd-9b4d-4b17-8c58-4edb7916762b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b"
Nov 02 22:30:10.646613 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:10.646591    5067 scope.go:122] "RemoveContainer" containerID="78f2fe3fb397a2096f084a68738fb67bb1b60221acbbcd22e1283d4d4cefa0a1"
Nov 02 22:30:10.646944 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:10.646919    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v_odh-ai-gateway-infra(b41ad331-309b-4899-98f6-5b313e554e0b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" podUID="b41ad331-309b-4899-98f6-5b313e554e0b"
Nov 02 22:30:10.648196 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:10.648172    5067 scope.go:122] "RemoveContainer" containerID="9b94cc09b6651333278e7fb9a01b805008392a9cb663a3070acb7742d248bab3"
Nov 02 22:30:10.648459 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:10.648437    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp_odh-ai-gateway-infra(5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4"
Nov 02 22:30:12.356193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.355733    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:30:12.365191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.363559    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:30:12.413117 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.413085    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:30:12.447245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447215    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-envoy\") pod \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") "
Nov 02 22:30:12.447456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447251    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-socket\") pod \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") "
Nov 02 22:30:12.447456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447292    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istiod-ca-cert\") pod \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") "
Nov 02 22:30:12.447456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447380    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-data\") pod \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") "
Nov 02 22:30:12.447456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447411    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-podinfo\") pod \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") "
Nov 02 22:30:12.447678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447466    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-credential-socket\") pod \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") "
Nov 02 22:30:12.447678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447516    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istiod-ca-cert\") pod \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") "
Nov 02 22:30:12.447678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447598    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-socket" pod "709169a6-5fe4-4f6b-a8ad-b1423a4bd939" (UID: "709169a6-5fe4-4f6b-a8ad-b1423a4bd939"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.447678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447616    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-socket\") pod \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") "
Nov 02 22:30:12.447678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447654    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-credential-socket\") pod \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") "
Nov 02 22:30:12.447998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447689    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-kube-api-access-927bl\" (UniqueName: \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-kube-api-access-927bl\") pod \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") "
Nov 02 22:30:12.447998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447728    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-data\") pod \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") "
Nov 02 22:30:12.447998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447761    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-token\" (UniqueName: \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-token\") pod \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") "
Nov 02 22:30:12.447998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447826    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-certs\") pod \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") "
Nov 02 22:30:12.447998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447870    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-data" pod "709169a6-5fe4-4f6b-a8ad-b1423a4bd939" (UID: "709169a6-5fe4-4f6b-a8ad-b1423a4bd939"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.447998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447879    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istiod-ca-cert" pod "709169a6-5fe4-4f6b-a8ad-b1423a4bd939" (UID: "709169a6-5fe4-4f6b-a8ad-b1423a4bd939"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:12.447998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447881    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-token\" (UniqueName: \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-token\") pod \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") "
Nov 02 22:30:12.447998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.447991    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-podinfo\") pod \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") "
Nov 02 22:30:12.448575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.448043    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istiod-ca-cert" pod "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" (UID: "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:12.448575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.448077    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-data" pod "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" (UID: "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.448575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.448099    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-credential-socket" pod "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" (UID: "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.448575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.448116    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-certs\") pod \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\" (UID: \"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7\") "
Nov 02 22:30:12.448575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.448185    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-envoy\") pod \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") "
Nov 02 22:30:12.448575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.448288    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-credential-socket" pod "709169a6-5fe4-4f6b-a8ad-b1423a4bd939" (UID: "709169a6-5fe4-4f6b-a8ad-b1423a4bd939"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.448575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.448303    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-kube-api-access-sk26n\" (UniqueName: \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-kube-api-access-sk26n\") pod \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\" (UID: \"709169a6-5fe4-4f6b-a8ad-b1423a4bd939\") "
Nov 02 22:30:12.448575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.448333    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-certs" pod "709169a6-5fe4-4f6b-a8ad-b1423a4bd939" (UID: "709169a6-5fe4-4f6b-a8ad-b1423a4bd939"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.449110 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.448602    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-socket" pod "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" (UID: "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.449110 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.448671    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-certs" pod "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" (UID: "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.449266 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.449181    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.449266 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.449205    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.449266 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.449221    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.449266 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.449236    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.449266 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.449251    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.449266 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.449266    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.449735 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.449281    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.449735 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.449298    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.449735 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.449312    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.449735 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.449326    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.450246 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.450170    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-envoy" pod "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" (UID: "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.450883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.450858    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-kube-api-access-927bl" pod "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" (UID: "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7"). InnerVolumeSpecName "kube-api-access-927bl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:12.451005 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.450896    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-token" pod "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" (UID: "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:12.451197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.451175    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-kube-api-access-sk26n" pod "709169a6-5fe4-4f6b-a8ad-b1423a4bd939" (UID: "709169a6-5fe4-4f6b-a8ad-b1423a4bd939"). InnerVolumeSpecName "kube-api-access-sk26n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:12.451347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.451323    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-envoy" pod "709169a6-5fe4-4f6b-a8ad-b1423a4bd939" (UID: "709169a6-5fe4-4f6b-a8ad-b1423a4bd939"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.451871 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.451850    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-podinfo" pod "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" (UID: "e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:30:12.451917 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.451847    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-podinfo" pod "709169a6-5fe4-4f6b-a8ad-b1423a4bd939" (UID: "709169a6-5fe4-4f6b-a8ad-b1423a4bd939"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:30:12.451993 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.451980    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-token" pod "709169a6-5fe4-4f6b-a8ad-b1423a4bd939" (UID: "709169a6-5fe4-4f6b-a8ad-b1423a4bd939"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:12.550417 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550382    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-kube-api-access-2cpgb\" (UniqueName: \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-kube-api-access-2cpgb\") pod \"a56fb386-b822-4107-b8ed-11c593c9a268\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") "
Nov 02 22:30:12.550620 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550508    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a56fb386-b822-4107-b8ed-11c593c9a268-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a56fb386-b822-4107-b8ed-11c593c9a268-istiod-ca-cert\") pod \"a56fb386-b822-4107-b8ed-11c593c9a268\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") "
Nov 02 22:30:12.550620 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550531    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-data\") pod \"a56fb386-b822-4107-b8ed-11c593c9a268\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") "
Nov 02 22:30:12.550747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550642    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-certs\") pod \"a56fb386-b822-4107-b8ed-11c593c9a268\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") "
Nov 02 22:30:12.550747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550676    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a56fb386-b822-4107-b8ed-11c593c9a268-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a56fb386-b822-4107-b8ed-11c593c9a268-istio-podinfo\") pod \"a56fb386-b822-4107-b8ed-11c593c9a268\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") "
Nov 02 22:30:12.550747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550716    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-credential-socket\") pod \"a56fb386-b822-4107-b8ed-11c593c9a268\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") "
Nov 02 22:30:12.551491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550767    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-socket\") pod \"a56fb386-b822-4107-b8ed-11c593c9a268\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") "
Nov 02 22:30:12.551491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550836    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-istio-token\" (UniqueName: \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-istio-token\") pod \"a56fb386-b822-4107-b8ed-11c593c9a268\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") "
Nov 02 22:30:12.551491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550900    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-envoy\") pod \"a56fb386-b822-4107-b8ed-11c593c9a268\" (UID: \"a56fb386-b822-4107-b8ed-11c593c9a268\") "
Nov 02 22:30:12.551491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550899    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-data" pod "a56fb386-b822-4107-b8ed-11c593c9a268" (UID: "a56fb386-b822-4107-b8ed-11c593c9a268"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.551491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550942    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-certs" pod "a56fb386-b822-4107-b8ed-11c593c9a268" (UID: "a56fb386-b822-4107-b8ed-11c593c9a268"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.551491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.550941    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a56fb386-b822-4107-b8ed-11c593c9a268-istiod-ca-cert" pod "a56fb386-b822-4107-b8ed-11c593c9a268" (UID: "a56fb386-b822-4107-b8ed-11c593c9a268"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:12.551491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551115    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-credential-socket" pod "a56fb386-b822-4107-b8ed-11c593c9a268" (UID: "a56fb386-b822-4107-b8ed-11c593c9a268"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.551491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551194    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-socket" pod "a56fb386-b822-4107-b8ed-11c593c9a268" (UID: "a56fb386-b822-4107-b8ed-11c593c9a268"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551696    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551717    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551732    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551744    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a56fb386-b822-4107-b8ed-11c593c9a268-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551757    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551769    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk26n\" (UniqueName: \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-kube-api-access-sk26n\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551782    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551795    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551807    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551819    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551831    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551844    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-927bl\" (UniqueName: \"kubernetes.io/projected/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7-kube-api-access-927bl\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.551945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.551856    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/709169a6-5fe4-4f6b-a8ad-b1423a4bd939-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.553145 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.553118    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a56fb386-b822-4107-b8ed-11c593c9a268-istio-podinfo" pod "a56fb386-b822-4107-b8ed-11c593c9a268" (UID: "a56fb386-b822-4107-b8ed-11c593c9a268"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:30:12.553525 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.553500    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-kube-api-access-2cpgb" pod "a56fb386-b822-4107-b8ed-11c593c9a268" (UID: "a56fb386-b822-4107-b8ed-11c593c9a268"). InnerVolumeSpecName "kube-api-access-2cpgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:12.553631 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.553562    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-istio-token" pod "a56fb386-b822-4107-b8ed-11c593c9a268" (UID: "a56fb386-b822-4107-b8ed-11c593c9a268"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:12.553803 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.553776    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-envoy" pod "a56fb386-b822-4107-b8ed-11c593c9a268" (UID: "a56fb386-b822-4107-b8ed-11c593c9a268"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:12.653558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.653468    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.653558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.653509    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a56fb386-b822-4107-b8ed-11c593c9a268-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.653558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.653527    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cpgb\" (UniqueName: \"kubernetes.io/projected/a56fb386-b822-4107-b8ed-11c593c9a268-kube-api-access-2cpgb\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.653558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.653543    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a56fb386-b822-4107-b8ed-11c593c9a268-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:12.671906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.668469    5067 generic.go:356] "Generic (PLEG): container finished" podID="a56fb386-b822-4107-b8ed-11c593c9a268" containerID="b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89" exitCode=0
Nov 02 22:30:12.671906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.668588    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" event={"ID":"a56fb386-b822-4107-b8ed-11c593c9a268","Type":"ContainerDied","Data":"b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89"}
Nov 02 22:30:12.671906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.668641    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7" event={"ID":"a56fb386-b822-4107-b8ed-11c593c9a268","Type":"ContainerDied","Data":"8341cb62523b8b25826f3be6cd8f8824a9ac8816d06b917a92b76d0f327d710d"}
Nov 02 22:30:12.671906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.668673    5067 scope.go:122] "RemoveContainer" containerID="b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89"
Nov 02 22:30:12.671906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.668806    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"
Nov 02 22:30:12.675809 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.675776    5067 generic.go:356] "Generic (PLEG): container finished" podID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerID="8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052" exitCode=0
Nov 02 22:30:12.676053 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.676031    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" event={"ID":"709169a6-5fe4-4f6b-a8ad-b1423a4bd939","Type":"ContainerDied","Data":"8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052"}
Nov 02 22:30:12.676125 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.676087    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx" event={"ID":"709169a6-5fe4-4f6b-a8ad-b1423a4bd939","Type":"ContainerDied","Data":"d8d30317b9d7bf464c884f8cc605d757c1442184658d9d8669f602b5bb2cc68b"}
Nov 02 22:30:12.676253 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.676172    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"
Nov 02 22:30:12.687750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.678905    5067 generic.go:356] "Generic (PLEG): container finished" podID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerID="1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798" exitCode=0
Nov 02 22:30:12.687750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.678990    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" event={"ID":"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7","Type":"ContainerDied","Data":"1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798"}
Nov 02 22:30:12.687750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.679029    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8" event={"ID":"e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7","Type":"ContainerDied","Data":"e637ce5cdc15ce1b51e735288f4cfc038c63a8aebe73f30496a1105db293d18e"}
Nov 02 22:30:12.687750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.679095    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"
Nov 02 22:30:12.692750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.691455    5067 scope.go:122] "RemoveContainer" containerID="b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89"
Nov 02 22:30:12.692750 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:12.692516    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89\": container with ID starting with b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89 not found: ID does not exist" containerID="b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89"
Nov 02 22:30:12.692750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.692556    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89"} err="failed to get container status \"b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89\": rpc error: code = NotFound desc = could not find container \"b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89\": container with ID starting with b92113bc825abbdb819c5bb98fe60c1af58123d503b79437d4d21c62b42bca89 not found: ID does not exist"
Nov 02 22:30:12.692750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.692576    5067 scope.go:122] "RemoveContainer" containerID="8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052"
Nov 02 22:30:12.703887 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.703862    5067 scope.go:122] "RemoveContainer" containerID="8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052"
Nov 02 22:30:12.704180 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:12.704160    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052\": container with ID starting with 8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052 not found: ID does not exist" containerID="8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052"
Nov 02 22:30:12.704278 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.704194    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052"} err="failed to get container status \"8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052\": rpc error: code = NotFound desc = could not find container \"8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052\": container with ID starting with 8976407476d84704d084a404ba435ece1d6423633c9224d0cd3e1192f0598052 not found: ID does not exist"
Nov 02 22:30:12.704278 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.704219    5067 scope.go:122] "RemoveContainer" containerID="1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798"
Nov 02 22:30:12.714377 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.714338    5067 scope.go:122] "RemoveContainer" containerID="1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798"
Nov 02 22:30:12.714688 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:12.714667    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798\": container with ID starting with 1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798 not found: ID does not exist" containerID="1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798"
Nov 02 22:30:12.714754 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.714699    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798"} err="failed to get container status \"1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798\": rpc error: code = NotFound desc = could not find container \"1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798\": container with ID starting with 1a76d3591b8ee740c20157866b490db583c12ca51376cff27c88f3b995c27798 not found: ID does not exist"
Nov 02 22:30:12.854660 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.854623    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"]
Nov 02 22:30:12.859324 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.859292    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-7t8z8"]
Nov 02 22:30:12.970489 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.970451    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"]
Nov 02 22:30:12.976750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:12.976715    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-g88r7"]
Nov 02 22:30:13.071335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:13.071297    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"]
Nov 02 22:30:13.075680 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:13.075653    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-711bd4-openshift-default-5d96df6c75-4qjdx"]
Nov 02 22:30:13.850144 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:13.850101    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" path="/var/lib/kubelet/pods/709169a6-5fe4-4f6b-a8ad-b1423a4bd939/volumes"
Nov 02 22:30:13.850900 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:13.850882    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" path="/var/lib/kubelet/pods/a56fb386-b822-4107-b8ed-11c593c9a268/volumes"
Nov 02 22:30:13.851570 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:13.851551    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" path="/var/lib/kubelet/pods/e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7/volumes"
Nov 02 22:30:14.213682 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.213594    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"]
Nov 02 22:30:14.213843 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.213828    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n" podUID="dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0" containerName="maas-api" containerID="cri-o://5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6" gracePeriod=30
Nov 02 22:30:14.654909 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.654875    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:30:14.703345 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.703312    5067 generic.go:356] "Generic (PLEG): container finished" podID="dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0" containerID="5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6" exitCode=0
Nov 02 22:30:14.703656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.703622    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n" event={"ID":"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0","Type":"ContainerDied","Data":"5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6"}
Nov 02 22:30:14.703743 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.703693    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n" event={"ID":"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0","Type":"ContainerDied","Data":"8f4fbc9fca6fa29464b5f91a5a16f280cadf8a6252ba25a1f791812fc3c3603e"}
Nov 02 22:30:14.703743 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.703726    5067 scope.go:122] "RemoveContainer" containerID="5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6"
Nov 02 22:30:14.703880 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.703865    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"
Nov 02 22:30:14.728866 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.728443    5067 scope.go:122] "RemoveContainer" containerID="5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6"
Nov 02 22:30:14.728987 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:14.728866    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6\": container with ID starting with 5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6 not found: ID does not exist" containerID="5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6"
Nov 02 22:30:14.728987 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.728900    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6"} err="failed to get container status \"5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6\": rpc error: code = NotFound desc = could not find container \"5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6\": container with ID starting with 5d7812437efaf68b1c0de599dfe0ef2f99b22b630c5ed026bc6159107b38eea6 not found: ID does not exist"
Nov 02 22:30:14.811384 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.807571    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-maas-api-tls\") pod \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") "
Nov 02 22:30:14.811384 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.807711    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-kube-api-access-s2z4n\" (UniqueName: \"kubernetes.io/projected/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-kube-api-access-s2z4n\") pod \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") "
Nov 02 22:30:14.811384 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.807756    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-openshift-service-ca\") pod \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") "
Nov 02 22:30:14.811384 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.807775    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-metrics-certs\") pod \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\" (UID: \"dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0\") "
Nov 02 22:30:14.811384 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.810851    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-metrics-certs" pod "dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0" (UID: "dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:30:14.813437 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.813302    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-maas-api-tls" pod "dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0" (UID: "dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:30:14.814511 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.814426    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-openshift-service-ca" pod "dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0" (UID: "dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:14.816293 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.816146    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-kube-api-access-s2z4n" pod "dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0" (UID: "dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0"). InnerVolumeSpecName "kube-api-access-s2z4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:14.909446 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.909301    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2z4n\" (UniqueName: \"kubernetes.io/projected/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-kube-api-access-s2z4n\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:14.909446 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.909335    5067 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-openshift-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:14.909446 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.909346    5067 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-metrics-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:14.909446 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:14.909355    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0-maas-api-tls\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:15.122551 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:15.122516    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"]
Nov 02 22:30:15.126583 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:15.126555    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7f1542e5-5f9f7b8d66-wjh7n"]
Nov 02 22:30:15.848929 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:15.848890    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0" path="/var/lib/kubelet/pods/dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0/volumes"
Nov 02 22:30:17.106968 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.105809    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"]
Nov 02 22:30:17.111609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111576    5067 cpu_manager.go:414] "containerMap: removing container" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerName="istio-proxy"
Nov 02 22:30:17.111609 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111606    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerName="istio-proxy"
Nov 02 22:30:17.111822 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111631    5067 cpu_manager.go:414] "containerMap: removing container" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" containerName="istio-proxy"
Nov 02 22:30:17.111822 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111640    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" containerName="istio-proxy"
Nov 02 22:30:17.111822 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111710    5067 cpu_manager.go:414] "containerMap: removing container" podUID="dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0" containerName="maas-api"
Nov 02 22:30:17.111822 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111717    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0" containerName="maas-api"
Nov 02 22:30:17.111822 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111744    5067 cpu_manager.go:414] "containerMap: removing container" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy"
Nov 02 22:30:17.111822 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111752    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy"
Nov 02 22:30:17.112142 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111923    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="709169a6-5fe4-4f6b-a8ad-b1423a4bd939" containerName="istio-proxy"
Nov 02 22:30:17.112142 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111941    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9e2ea44-ad21-46e7-b229-fe7d95bdc1a7" containerName="istio-proxy"
Nov 02 22:30:17.112142 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111953    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="a56fb386-b822-4107-b8ed-11c593c9a268" containerName="istio-proxy"
Nov 02 22:30:17.112142 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.111968    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbfd9b4a-d6cc-45d8-aa9b-dbe56dad9bb0" containerName="maas-api"
Nov 02 22:30:17.128595 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.125710    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.245473 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.238080    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bb1a7ce7-openshift-default-dockercfg-9f4ks\""
Nov 02 22:30:17.379268 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.379173    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"]
Nov 02 22:30:17.409234 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.407792    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-credential-socket\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.409234 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.407902    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-data\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.409234 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.407946    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-podinfo\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.409234 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.408001    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-envoy\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.409234 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.408027    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-token\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.409234 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.408072    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istiod-ca-cert\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.409234 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.408292    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-certs\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.409234 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.408358    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-socket\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.409234 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.408536    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgfm2\" (UniqueName: \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-kube-api-access-cgfm2\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.569582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.563134    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-envoy\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.569582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.563194    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-token\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.569582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.563231    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istiod-ca-cert\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.569582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.563419    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-certs\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.569582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.563473    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-socket\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.569582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.564463    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istiod-ca-cert\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.569582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.565124    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-certs\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.569582 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.569488    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-socket\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.576385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.572514    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgfm2\" (UniqueName: \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-kube-api-access-cgfm2\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.576385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.572633    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-credential-socket\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.576385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.572711    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-data\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.576385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.572744    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-podinfo\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.576385 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.575466    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-credential-socket\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.579943 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.579890    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-envoy\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.588490 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.582271    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-data\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.610767 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.608788    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-podinfo\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.626524 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.626488    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-token\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.651339 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.651211    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgfm2\" (UniqueName: \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-kube-api-access-cgfm2\") pod \"e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") " pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:17.763387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:17.760918    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:18.303993 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:18.303962    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"]
Nov 02 22:30:18.305095 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:30:18.305068    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da39c9c_fdc0_4899_9dd5_9ea4ab06ed01.slice/crio-e5f381c7fdee6b54583f9bf1bef91fecde170a5962ccab9d621e7e94c1ce5bfc WatchSource:0}: Error finding container e5f381c7fdee6b54583f9bf1bef91fecde170a5962ccab9d621e7e94c1ce5bfc: Status 404 returned error can't find the container with id e5f381c7fdee6b54583f9bf1bef91fecde170a5962ccab9d621e7e94c1ce5bfc
Nov 02 22:30:18.307665 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:18.307629    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:30:18.307782 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:18.307715    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:30:18.307782 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:18.307762    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:30:18.317148 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:18.317112    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"]
Nov 02 22:30:18.841034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:18.839873    5067 scope.go:122] "RemoveContainer" containerID="9b94cc09b6651333278e7fb9a01b805008392a9cb663a3070acb7742d248bab3"
Nov 02 22:30:18.880397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:18.873048    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" event={"ID":"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01","Type":"ContainerStarted","Data":"aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6"}
Nov 02 22:30:18.880397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:18.873120    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" event={"ID":"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01","Type":"ContainerStarted","Data":"e5f381c7fdee6b54583f9bf1bef91fecde170a5962ccab9d621e7e94c1ce5bfc"}
Nov 02 22:30:19.019346 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:19.019267    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" podStartSLOduration=2.01922979 podStartE2EDuration="2.01922979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:30:18.996043397 +0000 UTC m=+1491.279853085" watchObservedRunningTime="2026-11-02 22:30:19.01922979 +0000 UTC m=+1491.303039479"
Nov 02 22:30:19.770219 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:19.767760    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:19.829476 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:19.829443    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:19.933144 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:19.933100    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" event={"ID":"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4","Type":"ContainerStarted","Data":"248fddf3aef9230cab5ab6b84807a215792a52584b9665a46982563c161a9791"}
Nov 02 22:30:19.933653 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:19.933604    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:19.935750 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:19.935716    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:30:20.838849 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:20.838809    5067 scope.go:122] "RemoveContainer" containerID="e18ef49b68b00f381ccf0a720bf87d8851f7b50c96caa15d4690c1c7d405e239"
Nov 02 22:30:20.944165 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:20.944127    5067 generic.go:356] "Generic (PLEG): container finished" podID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerID="248fddf3aef9230cab5ab6b84807a215792a52584b9665a46982563c161a9791" exitCode=60
Nov 02 22:30:20.944343 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:20.944206    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" event={"ID":"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4","Type":"ContainerDied","Data":"248fddf3aef9230cab5ab6b84807a215792a52584b9665a46982563c161a9791"}
Nov 02 22:30:20.944343 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:20.944286    5067 scope.go:122] "RemoveContainer" containerID="9b94cc09b6651333278e7fb9a01b805008392a9cb663a3070acb7742d248bab3"
Nov 02 22:30:20.945155 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:20.945124    5067 scope.go:122] "RemoveContainer" containerID="248fddf3aef9230cab5ab6b84807a215792a52584b9665a46982563c161a9791"
Nov 02 22:30:20.945566 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:20.945539    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp_odh-ai-gateway-infra(5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4"
Nov 02 22:30:21.070388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:21.070334    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"]
Nov 02 22:30:21.583749 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:21.583715    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"]
Nov 02 22:30:21.583956 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:21.583931    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq" podUID="71a8dcab-fda2-4e49-9258-c7f701958f32" containerName="maas-api" containerID="cri-o://13471363b308d7bb63cfc1e2b63ec48e6439964df5a7b33268fc5d17ef51ab81" gracePeriod=30
Nov 02 22:30:21.880388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:21.877059    5067 scope.go:122] "RemoveContainer" containerID="78f2fe3fb397a2096f084a68738fb67bb1b60221acbbcd22e1283d4d4cefa0a1"
Nov 02 22:30:21.988929 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:21.988853    5067 generic.go:356] "Generic (PLEG): container finished" podID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerID="46794222faad12644b17060152604d9de5dabe0fae33d9610a06aed79f4cfed9" exitCode=60
Nov 02 22:30:21.991553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:21.991524    5067 generic.go:356] "Generic (PLEG): container finished" podID="71a8dcab-fda2-4e49-9258-c7f701958f32" containerID="13471363b308d7bb63cfc1e2b63ec48e6439964df5a7b33268fc5d17ef51ab81" exitCode=0
Nov 02 22:30:22.020839 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.020803    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v" event={"ID":"b9bed3dd-9b4d-4b17-8c58-4edb7916762b","Type":"ContainerDied","Data":"46794222faad12644b17060152604d9de5dabe0fae33d9610a06aed79f4cfed9"}
Nov 02 22:30:22.020999 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.020878    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq" event={"ID":"71a8dcab-fda2-4e49-9258-c7f701958f32","Type":"ContainerDied","Data":"13471363b308d7bb63cfc1e2b63ec48e6439964df5a7b33268fc5d17ef51ab81"}
Nov 02 22:30:22.020999 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.020914    5067 scope.go:122] "RemoveContainer" containerID="e18ef49b68b00f381ccf0a720bf87d8851f7b50c96caa15d4690c1c7d405e239"
Nov 02 22:30:22.021736 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.021716    5067 scope.go:122] "RemoveContainer" containerID="46794222faad12644b17060152604d9de5dabe0fae33d9610a06aed79f4cfed9"
Nov 02 22:30:22.022309 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:22.022281    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v_odh-ai-gateway-infra(b9bed3dd-9b4d-4b17-8c58-4edb7916762b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b"
Nov 02 22:30:22.077945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.077909    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:30:22.204399 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.200010    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-metrics-certs\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-metrics-certs\") pod \"71a8dcab-fda2-4e49-9258-c7f701958f32\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") "
Nov 02 22:30:22.229425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.217703    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71a8dcab-fda2-4e49-9258-c7f701958f32-kube-api-access-5zqb4\" (UniqueName: \"kubernetes.io/projected/71a8dcab-fda2-4e49-9258-c7f701958f32-kube-api-access-5zqb4\") pod \"71a8dcab-fda2-4e49-9258-c7f701958f32\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") "
Nov 02 22:30:22.235476 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.230205    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-maas-api-tls\") pod \"71a8dcab-fda2-4e49-9258-c7f701958f32\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") "
Nov 02 22:30:22.362154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.330418    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71a8dcab-fda2-4e49-9258-c7f701958f32-openshift-service-ca" pod "71a8dcab-fda2-4e49-9258-c7f701958f32" (UID: "71a8dcab-fda2-4e49-9258-c7f701958f32"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:22.362154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.336951    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71a8dcab-fda2-4e49-9258-c7f701958f32-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a8dcab-fda2-4e49-9258-c7f701958f32-openshift-service-ca\") pod \"71a8dcab-fda2-4e49-9258-c7f701958f32\" (UID: \"71a8dcab-fda2-4e49-9258-c7f701958f32\") "
Nov 02 22:30:22.362154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.337901    5067 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71a8dcab-fda2-4e49-9258-c7f701958f32-openshift-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:22.362154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.339736    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-metrics-certs" pod "71a8dcab-fda2-4e49-9258-c7f701958f32" (UID: "71a8dcab-fda2-4e49-9258-c7f701958f32"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:30:22.362154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.362011    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-maas-api-tls" pod "71a8dcab-fda2-4e49-9258-c7f701958f32" (UID: "71a8dcab-fda2-4e49-9258-c7f701958f32"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:30:22.363025 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.362971    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a8dcab-fda2-4e49-9258-c7f701958f32-kube-api-access-5zqb4" pod "71a8dcab-fda2-4e49-9258-c7f701958f32" (UID: "71a8dcab-fda2-4e49-9258-c7f701958f32"). InnerVolumeSpecName "kube-api-access-5zqb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:22.444102 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.444062    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zqb4\" (UniqueName: \"kubernetes.io/projected/71a8dcab-fda2-4e49-9258-c7f701958f32-kube-api-access-5zqb4\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:22.444332 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.444313    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-maas-api-tls\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:22.444448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.444434    5067 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71a8dcab-fda2-4e49-9258-c7f701958f32-metrics-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:22.510865 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.510524    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"
Nov 02 22:30:22.547508 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.547475    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-kube-api-access-5fngx\" (UniqueName: \"kubernetes.io/projected/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-kube-api-access-5fngx\") pod \"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4\" (UID: \"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4\") "
Nov 02 22:30:22.547660 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.547607    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-maas-api-service-ca\") pod \"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4\" (UID: \"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4\") "
Nov 02 22:30:22.548053 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.548026    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-maas-api-service-ca" pod "5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" (UID: "5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:22.549593 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.549572    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-kube-api-access-5fngx" pod "5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" (UID: "5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4"). InnerVolumeSpecName "kube-api-access-5fngx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:22.649354 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.649274    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fngx\" (UniqueName: \"kubernetes.io/projected/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-kube-api-access-5fngx\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:22.649354 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:22.649312    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4-maas-api-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:23.004546 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.004510    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"
Nov 02 22:30:23.004993 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.004546    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq" event={"ID":"71a8dcab-fda2-4e49-9258-c7f701958f32","Type":"ContainerDied","Data":"c2a6c426ff0c6d09a1a59f58d5be407e31a007dd7274dc0c0b0f4721ef3fd8e1"}
Nov 02 22:30:23.004993 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.004615    5067 scope.go:122] "RemoveContainer" containerID="13471363b308d7bb63cfc1e2b63ec48e6439964df5a7b33268fc5d17ef51ab81"
Nov 02 22:30:23.006863 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.006833    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" event={"ID":"b41ad331-309b-4899-98f6-5b313e554e0b","Type":"ContainerStarted","Data":"7681acceaecec2c375181831b92fa5fcae8eb625c55697856e9889fb87630a93"}
Nov 02 22:30:23.008466 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.008436    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"
Nov 02 22:30:23.008604 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.008469    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp" event={"ID":"5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4","Type":"ContainerDied","Data":"0bb465a6d65635f6c86cc7b46b0085a611466b9f620a6bcfe7cec089aa8cdb7c"}
Nov 02 22:30:23.016174 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.016152    5067 scope.go:122] "RemoveContainer" containerID="248fddf3aef9230cab5ab6b84807a215792a52584b9665a46982563c161a9791"
Nov 02 22:30:23.239282 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.239246    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"]
Nov 02 22:30:23.272577 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.272484    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"]
Nov 02 22:30:23.279456 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.279420    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-2e86dfb1-29894310-4vqsp"]
Nov 02 22:30:23.317689 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.317646    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv" podUID="7f966b0b-b093-439f-ba90-be6684b0f1ce" containerName="payload-pre-processing" containerID="cri-o://5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e" gracePeriod=25
Nov 02 22:30:23.413715 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.413687    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"]
Nov 02 22:30:23.437136 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.437100    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-db36b079-56967d848d-rbhwq"]
Nov 02 22:30:23.449825 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.449802    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"
Nov 02 22:30:23.467465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.467441    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-kube-api-access-6bs92\" (UniqueName: \"kubernetes.io/projected/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-kube-api-access-6bs92\") pod \"b9bed3dd-9b4d-4b17-8c58-4edb7916762b\" (UID: \"b9bed3dd-9b4d-4b17-8c58-4edb7916762b\") "
Nov 02 22:30:23.467598 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.467565    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-maas-api-service-ca\") pod \"b9bed3dd-9b4d-4b17-8c58-4edb7916762b\" (UID: \"b9bed3dd-9b4d-4b17-8c58-4edb7916762b\") "
Nov 02 22:30:23.468058 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.468032    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-maas-api-service-ca" pod "b9bed3dd-9b4d-4b17-8c58-4edb7916762b" (UID: "b9bed3dd-9b4d-4b17-8c58-4edb7916762b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:23.468356 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.468341    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-maas-api-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:23.469739 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.469718    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-kube-api-access-6bs92" pod "b9bed3dd-9b4d-4b17-8c58-4edb7916762b" (UID: "b9bed3dd-9b4d-4b17-8c58-4edb7916762b"). InnerVolumeSpecName "kube-api-access-6bs92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:23.569648 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.569565    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bs92\" (UniqueName: \"kubernetes.io/projected/b9bed3dd-9b4d-4b17-8c58-4edb7916762b-kube-api-access-6bs92\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:23.633607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.633581    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:30:23.670404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.670351    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f966b0b-b093-439f-ba90-be6684b0f1ce-kube-api-access-nvxtt\" (UniqueName: \"kubernetes.io/projected/7f966b0b-b093-439f-ba90-be6684b0f1ce-kube-api-access-nvxtt\") pod \"7f966b0b-b093-439f-ba90-be6684b0f1ce\" (UID: \"7f966b0b-b093-439f-ba90-be6684b0f1ce\") "
Nov 02 22:30:23.670573 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.670434    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f966b0b-b093-439f-ba90-be6684b0f1ce-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f966b0b-b093-439f-ba90-be6684b0f1ce-plugins-config-volume\") pod \"7f966b0b-b093-439f-ba90-be6684b0f1ce\" (UID: \"7f966b0b-b093-439f-ba90-be6684b0f1ce\") "
Nov 02 22:30:23.671766 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.671734    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f966b0b-b093-439f-ba90-be6684b0f1ce-plugins-config-volume" pod "7f966b0b-b093-439f-ba90-be6684b0f1ce" (UID: "7f966b0b-b093-439f-ba90-be6684b0f1ce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:23.674121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.674092    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f966b0b-b093-439f-ba90-be6684b0f1ce-kube-api-access-nvxtt" pod "7f966b0b-b093-439f-ba90-be6684b0f1ce" (UID: "7f966b0b-b093-439f-ba90-be6684b0f1ce"). InnerVolumeSpecName "kube-api-access-nvxtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:23.773131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.773090    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvxtt\" (UniqueName: \"kubernetes.io/projected/7f966b0b-b093-439f-ba90-be6684b0f1ce-kube-api-access-nvxtt\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:23.773131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.773128    5067 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f966b0b-b093-439f-ba90-be6684b0f1ce-plugins-config-volume\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:23.846291 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.846212    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" path="/var/lib/kubelet/pods/5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4/volumes"
Nov 02 22:30:23.846920 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:23.846897    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71a8dcab-fda2-4e49-9258-c7f701958f32" path="/var/lib/kubelet/pods/71a8dcab-fda2-4e49-9258-c7f701958f32/volumes"
Nov 02 22:30:24.020865 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.020830    5067 generic.go:356] "Generic (PLEG): container finished" podID="7f966b0b-b093-439f-ba90-be6684b0f1ce" containerID="5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e" exitCode=0
Nov 02 22:30:24.021329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.020931    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv" event={"ID":"7f966b0b-b093-439f-ba90-be6684b0f1ce","Type":"ContainerDied","Data":"5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e"}
Nov 02 22:30:24.021329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.020969    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv" event={"ID":"7f966b0b-b093-439f-ba90-be6684b0f1ce","Type":"ContainerDied","Data":"cc785c7829bfc4acab8860e1510125511b26ada9e89b55277f4c21c8430a9e7c"}
Nov 02 22:30:24.021329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.020934    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"
Nov 02 22:30:24.021329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.020991    5067 scope.go:122] "RemoveContainer" containerID="5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e"
Nov 02 22:30:24.022538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.022523    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"
Nov 02 22:30:24.022660 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.022549    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v" event={"ID":"b9bed3dd-9b4d-4b17-8c58-4edb7916762b","Type":"ContainerDied","Data":"bfed4ff1f3fe7bc54f756bd57b70a1f6935b16a21a71e49155070ecd74f07bc6"}
Nov 02 22:30:24.026273 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.026250    5067 generic.go:356] "Generic (PLEG): container finished" podID="b41ad331-309b-4899-98f6-5b313e554e0b" containerID="7681acceaecec2c375181831b92fa5fcae8eb625c55697856e9889fb87630a93" exitCode=60
Nov 02 22:30:24.026424 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.026408    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" event={"ID":"b41ad331-309b-4899-98f6-5b313e554e0b","Type":"ContainerDied","Data":"7681acceaecec2c375181831b92fa5fcae8eb625c55697856e9889fb87630a93"}
Nov 02 22:30:24.026845 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.026821    5067 scope.go:122] "RemoveContainer" containerID="7681acceaecec2c375181831b92fa5fcae8eb625c55697856e9889fb87630a93"
Nov 02 22:30:24.027146 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:24.027122    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v_odh-ai-gateway-infra(b41ad331-309b-4899-98f6-5b313e554e0b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" podUID="b41ad331-309b-4899-98f6-5b313e554e0b"
Nov 02 22:30:24.031798 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.031780    5067 scope.go:122] "RemoveContainer" containerID="5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e"
Nov 02 22:30:24.032044 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:24.032017    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e\": container with ID starting with 5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e not found: ID does not exist" containerID="5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e"
Nov 02 22:30:24.032099 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.032044    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e"} err="failed to get container status \"5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e\": rpc error: code = NotFound desc = could not find container \"5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e\": container with ID starting with 5893fe11d9a478a056f60ee2d2a4b79fb87eb14ea811bce4da43cfbd026ed58e not found: ID does not exist"
Nov 02 22:30:24.032099 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.032059    5067 scope.go:122] "RemoveContainer" containerID="46794222faad12644b17060152604d9de5dabe0fae33d9610a06aed79f4cfed9"
Nov 02 22:30:24.042199 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.042183    5067 scope.go:122] "RemoveContainer" containerID="78f2fe3fb397a2096f084a68738fb67bb1b60221acbbcd22e1283d4d4cefa0a1"
Nov 02 22:30:24.192449 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.178180    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"]
Nov 02 22:30:24.344390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.339839    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"]
Nov 02 22:30:24.344390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.342859    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8263a4-29894310-zlc6v"]
Nov 02 22:30:24.456496 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.449576    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"]
Nov 02 22:30:24.475421 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:24.468308    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e86dfb1-56564966d7-l85zv"]
Nov 02 22:30:25.375425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:25.372046    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"
Nov 02 22:30:25.421008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:25.419718    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b41ad331-309b-4899-98f6-5b313e554e0b-kube-api-access-xmng7\" (UniqueName: \"kubernetes.io/projected/b41ad331-309b-4899-98f6-5b313e554e0b-kube-api-access-xmng7\") pod \"b41ad331-309b-4899-98f6-5b313e554e0b\" (UID: \"b41ad331-309b-4899-98f6-5b313e554e0b\") "
Nov 02 22:30:25.421008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:25.419797    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b41ad331-309b-4899-98f6-5b313e554e0b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b41ad331-309b-4899-98f6-5b313e554e0b-maas-api-service-ca\") pod \"b41ad331-309b-4899-98f6-5b313e554e0b\" (UID: \"b41ad331-309b-4899-98f6-5b313e554e0b\") "
Nov 02 22:30:25.421008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:25.420731    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b41ad331-309b-4899-98f6-5b313e554e0b-maas-api-service-ca" pod "b41ad331-309b-4899-98f6-5b313e554e0b" (UID: "b41ad331-309b-4899-98f6-5b313e554e0b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:25.425188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:25.425165    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b41ad331-309b-4899-98f6-5b313e554e0b-kube-api-access-xmng7" pod "b41ad331-309b-4899-98f6-5b313e554e0b" (UID: "b41ad331-309b-4899-98f6-5b313e554e0b"). InnerVolumeSpecName "kube-api-access-xmng7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:25.522735 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:25.522705    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmng7\" (UniqueName: \"kubernetes.io/projected/b41ad331-309b-4899-98f6-5b313e554e0b-kube-api-access-xmng7\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:25.522985 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:25.522965    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b41ad331-309b-4899-98f6-5b313e554e0b-maas-api-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:25.995392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:25.987865    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f966b0b-b093-439f-ba90-be6684b0f1ce" path="/var/lib/kubelet/pods/7f966b0b-b093-439f-ba90-be6684b0f1ce/volumes"
Nov 02 22:30:26.010388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.009557    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" path="/var/lib/kubelet/pods/b9bed3dd-9b4d-4b17-8c58-4edb7916762b/volumes"
Nov 02 22:30:26.129389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.122255    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v" event={"ID":"b41ad331-309b-4899-98f6-5b313e554e0b","Type":"ContainerDied","Data":"be9d1264715c3352d8204908caace921567fe4d28ecaa801b2732792e9640ddd"}
Nov 02 22:30:26.129389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.122327    5067 scope.go:122] "RemoveContainer" containerID="7681acceaecec2c375181831b92fa5fcae8eb625c55697856e9889fb87630a93"
Nov 02 22:30:26.129389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.122436    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"
Nov 02 22:30:26.354391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.350621    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"]
Nov 02 22:30:26.406263 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.396798    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-fe810f-29894310-dz66v"]
Nov 02 22:30:26.692447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.691571    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-mxjzx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:30:26.692447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.691659    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" podUID="f7a52edf-7fef-427a-99cd-4add52eaf196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.964728    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"]
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965670    5067 cpu_manager.go:414] "containerMap: removing container" podUID="71a8dcab-fda2-4e49-9258-c7f701958f32" containerName="maas-api"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965693    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71a8dcab-fda2-4e49-9258-c7f701958f32" containerName="maas-api"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965714    5067 cpu_manager.go:414] "containerMap: removing container" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965722    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965751    5067 cpu_manager.go:414] "containerMap: removing container" podUID="b41ad331-309b-4899-98f6-5b313e554e0b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965759    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b41ad331-309b-4899-98f6-5b313e554e0b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965775    5067 cpu_manager.go:414] "containerMap: removing container" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965783    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965797    5067 cpu_manager.go:414] "containerMap: removing container" podUID="b41ad331-309b-4899-98f6-5b313e554e0b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965804    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b41ad331-309b-4899-98f6-5b313e554e0b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965835    5067 cpu_manager.go:414] "containerMap: removing container" podUID="7f966b0b-b093-439f-ba90-be6684b0f1ce" containerName="payload-pre-processing"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965843    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f966b0b-b093-439f-ba90-be6684b0f1ce" containerName="payload-pre-processing"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965855    5067 cpu_manager.go:414] "containerMap: removing container" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965862    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965873    5067 cpu_manager.go:414] "containerMap: removing container" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965881    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965911    5067 cpu_manager.go:414] "containerMap: removing container" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965918    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965932    5067 cpu_manager.go:414] "containerMap: removing container" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.965938    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.966074    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f966b0b-b093-439f-ba90-be6684b0f1ce" containerName="payload-pre-processing"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.966087    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.966096    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.966110    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.966120    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="b41ad331-309b-4899-98f6-5b313e554e0b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.966128    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="5290c2f6-d0f8-4a67-aa29-f2b3fb63cda4" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.966138    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="71a8dcab-fda2-4e49-9258-c7f701958f32" containerName="maas-api"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.966146    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.966158    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="b41ad331-309b-4899-98f6-5b313e554e0b" containerName="cleanup"
Nov 02 22:30:26.968442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.966167    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="b41ad331-309b-4899-98f6-5b313e554e0b" containerName="cleanup"
Nov 02 22:30:26.978017 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:26.977990    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.043411 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.039801    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fb670683-openshift-default-dockercfg-fkhhb\""
Nov 02 22:30:27.043411 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.042234    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"]
Nov 02 22:30:27.064388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.049358    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95b6e129-f57a-4705-915b-52e87773e831-istio-podinfo\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.064388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.049439    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-data\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.064388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.049491    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-envoy\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.064388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.049577    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-istio-token\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.064388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.049626    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95b6e129-f57a-4705-915b-52e87773e831-istiod-ca-cert\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.064388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.049683    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-socket\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.064388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.049713    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc4dj\" (UniqueName: \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-kube-api-access-hc4dj\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.064388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.049814    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-certs\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.064388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.049849    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-credential-socket\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.174389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.168215    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-socket\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.174389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.168277    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc4dj\" (UniqueName: \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-kube-api-access-hc4dj\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.179776    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-certs\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.179849    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-credential-socket\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.179995    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95b6e129-f57a-4705-915b-52e87773e831-istio-podinfo\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.180038    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-data\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.180103    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-envoy\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.180229    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-istio-token\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.180293    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95b6e129-f57a-4705-915b-52e87773e831-istiod-ca-cert\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.181178    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95b6e129-f57a-4705-915b-52e87773e831-istiod-ca-cert\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.181806    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-socket\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.182292    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-certs\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.187390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.182522    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-credential-socket\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.200793 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.198829    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-data\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.265473 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.264586    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-envoy\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.277389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.266157    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95b6e129-f57a-4705-915b-52e87773e831-istio-podinfo\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.278256 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.278221    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:30:27.278608 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.278574    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:30:27.393460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.393389    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-istio-token\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.453284 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.452169    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc4dj\" (UniqueName: \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-kube-api-access-hc4dj\") pod \"e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") " pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.654468 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.653475    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:27.897391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:27.896745    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b41ad331-309b-4899-98f6-5b313e554e0b" path="/var/lib/kubelet/pods/b41ad331-309b-4899-98f6-5b313e554e0b/volumes"
Nov 02 22:30:28.346139 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:28.344944    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:30:28.395388 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:28.395153    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"]
Nov 02 22:30:28.403505 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:30:28.403477    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b6e129_f57a_4705_915b_52e87773e831.slice/crio-e8a6df517dea6ba58323a980c32a56f3600782bad3dcf8c7de7e7dbc9abb29ec WatchSource:0}: Error finding container e8a6df517dea6ba58323a980c32a56f3600782bad3dcf8c7de7e7dbc9abb29ec: Status 404 returned error can't find the container with id e8a6df517dea6ba58323a980c32a56f3600782bad3dcf8c7de7e7dbc9abb29ec
Nov 02 22:30:28.406162 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:28.406141    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:30:28.406755 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:28.406720    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:30:28.406849 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:28.406803    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:30:28.406920 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:28.406846    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:30:29.267742 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:29.267698    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" event={"ID":"95b6e129-f57a-4705-915b-52e87773e831","Type":"ContainerStarted","Data":"42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880"}
Nov 02 22:30:29.268211 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:29.267798    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" event={"ID":"95b6e129-f57a-4705-915b-52e87773e831","Type":"ContainerStarted","Data":"e8a6df517dea6ba58323a980c32a56f3600782bad3dcf8c7de7e7dbc9abb29ec"}
Nov 02 22:30:29.432176 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:29.432112    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" podStartSLOduration=3.43209775 podStartE2EDuration="3.43209775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:30:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:30:29.429575926 +0000 UTC m=+1501.713385612" watchObservedRunningTime="2026-11-02 22:30:29.43209775 +0000 UTC m=+1501.715907438"
Nov 02 22:30:29.654837 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:29.654750    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:29.667621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:29.667591    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:30.275450 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:30.275414    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:30.276587 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:30.276564    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:30:37.577150 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:37.577114    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:30:39.140507 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.140469    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"]
Nov 02 22:30:39.141189 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.141169    5067 cpu_manager.go:414] "containerMap: removing container" podUID="b41ad331-309b-4899-98f6-5b313e554e0b" containerName="cleanup"
Nov 02 22:30:39.141189 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.141186    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b41ad331-309b-4899-98f6-5b313e554e0b" containerName="cleanup"
Nov 02 22:30:39.141343 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.141290    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9bed3dd-9b4d-4b17-8c58-4edb7916762b" containerName="cleanup"
Nov 02 22:30:39.146872 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.146855    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.195775 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.195740    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 02 22:30:39.196034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.196004    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 02 22:30:39.196125 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.196040    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 02 22:30:39.196775 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.196754    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.196837 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.196795    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.196902 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.196886    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.196940 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.196922    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.196973 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.196955    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.197191 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.197174    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv8dp\" (UniqueName: \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-kube-api-access-qv8dp\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.197312 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.197295    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.242067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.242035    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-p7cs8\""
Nov 02 22:30:39.248227 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.248189    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"]
Nov 02 22:30:39.299452 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.299410    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.299668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.299529    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.299668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.299551    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.299668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.299589    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.299668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.299629    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.299916 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.299670    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.299916 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.299816    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8dp\" (UniqueName: \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-kube-api-access-qv8dp\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.300330 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.300302    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.302939 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.302911    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.304003 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.303977    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.305129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.305103    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.305237 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.305179    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.358345 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.358312    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.358535 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.358380    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv8dp\" (UniqueName: \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-kube-api-access-qv8dp\") pod \"istiod-openshift-gateway-7ddd9cfdc8-np74v\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.459691 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.459597    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:39.887760 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:39.887734    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"]
Nov 02 22:30:39.889405 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:30:39.889332    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8004d9_e8cc_4e69_b78b_8c9d74c2e907.slice/crio-723a0a5d538392890306bc1eb48e3d2cd8871db4c6aae0364f7c0c70d93af43e WatchSource:0}: Error finding container 723a0a5d538392890306bc1eb48e3d2cd8871db4c6aae0364f7c0c70d93af43e: Status 404 returned error can't find the container with id 723a0a5d538392890306bc1eb48e3d2cd8871db4c6aae0364f7c0c70d93af43e
Nov 02 22:30:40.344240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:40.344202    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v" event={"ID":"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907","Type":"ContainerStarted","Data":"723a0a5d538392890306bc1eb48e3d2cd8871db4c6aae0364f7c0c70d93af43e"}
Nov 02 22:30:44.344075 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:44.344037    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:30:44.344533 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:44.344102    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:30:45.393074 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:45.393018    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v" event={"ID":"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907","Type":"ContainerStarted","Data":"e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721"}
Nov 02 22:30:45.393598 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:45.393103    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:45.397661 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:45.395686    5067 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-np74v container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:30:45.397661 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:45.395776    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v" podUID="1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:30:45.590316 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:45.590260    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v" podStartSLOduration=2.1390706919999998 podStartE2EDuration="6.590244517s" totalImagesPullingTime="4.451173825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:30:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:30:45.552759963 +0000 UTC m=+1517.836569680" watchObservedRunningTime="2026-11-02 22:30:45.590244517 +0000 UTC m=+1517.874054203"
Nov 02 22:30:46.380787 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:46.380749    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"]
Nov 02 22:30:46.381084 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:46.381051    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy" containerID="cri-o://fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4" gracePeriod=30
Nov 02 22:30:46.392315 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:46.392281    5067 patch_prober.go:29] interesting pod/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:30:46.392521 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:46.392357    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:30:46.402449 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:46.402425    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:30:51.710810 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.710777    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:30:51.740190 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740142    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-data\") pod \"8881e503-831c-4519-b1b1-75312e378a79\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") "
Nov 02 22:30:51.740348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740282    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-kube-api-access-x959x\" (UniqueName: \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-kube-api-access-x959x\") pod \"8881e503-831c-4519-b1b1-75312e378a79\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") "
Nov 02 22:30:51.740487 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740459    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-data" pod "8881e503-831c-4519-b1b1-75312e378a79" (UID: "8881e503-831c-4519-b1b1-75312e378a79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:51.740528 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740481    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-istio-token\" (UniqueName: \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-istio-token\") pod \"8881e503-831c-4519-b1b1-75312e378a79\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") "
Nov 02 22:30:51.740563 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740524    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8881e503-831c-4519-b1b1-75312e378a79-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8881e503-831c-4519-b1b1-75312e378a79-istiod-ca-cert\") pod \"8881e503-831c-4519-b1b1-75312e378a79\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") "
Nov 02 22:30:51.740669 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740651    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-envoy\") pod \"8881e503-831c-4519-b1b1-75312e378a79\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") "
Nov 02 22:30:51.740747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740706    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8881e503-831c-4519-b1b1-75312e378a79-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8881e503-831c-4519-b1b1-75312e378a79-istio-podinfo\") pod \"8881e503-831c-4519-b1b1-75312e378a79\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") "
Nov 02 22:30:51.740813 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740789    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-credential-socket\") pod \"8881e503-831c-4519-b1b1-75312e378a79\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") "
Nov 02 22:30:51.740866 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740818    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-socket\") pod \"8881e503-831c-4519-b1b1-75312e378a79\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") "
Nov 02 22:30:51.740918 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740873    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8881e503-831c-4519-b1b1-75312e378a79-istiod-ca-cert" pod "8881e503-831c-4519-b1b1-75312e378a79" (UID: "8881e503-831c-4519-b1b1-75312e378a79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:30:51.740975 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.740916    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-certs\") pod \"8881e503-831c-4519-b1b1-75312e378a79\" (UID: \"8881e503-831c-4519-b1b1-75312e378a79\") "
Nov 02 22:30:51.741075 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.741031    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-credential-socket" pod "8881e503-831c-4519-b1b1-75312e378a79" (UID: "8881e503-831c-4519-b1b1-75312e378a79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:51.741210 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.741182    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-socket" pod "8881e503-831c-4519-b1b1-75312e378a79" (UID: "8881e503-831c-4519-b1b1-75312e378a79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:51.741433 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.741254    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-certs" pod "8881e503-831c-4519-b1b1-75312e378a79" (UID: "8881e503-831c-4519-b1b1-75312e378a79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:51.741757 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.741737    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8881e503-831c-4519-b1b1-75312e378a79-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:51.741854 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.741762    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:51.741854 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.741777    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:51.741854 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.741790    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:51.741854 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.741804    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:51.743412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.743346    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-istio-token" pod "8881e503-831c-4519-b1b1-75312e378a79" (UID: "8881e503-831c-4519-b1b1-75312e378a79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:51.743592 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.743568    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8881e503-831c-4519-b1b1-75312e378a79-istio-podinfo" pod "8881e503-831c-4519-b1b1-75312e378a79" (UID: "8881e503-831c-4519-b1b1-75312e378a79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:30:51.743679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.743595    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-envoy" pod "8881e503-831c-4519-b1b1-75312e378a79" (UID: "8881e503-831c-4519-b1b1-75312e378a79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:30:51.743841 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.743818    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-kube-api-access-x959x" pod "8881e503-831c-4519-b1b1-75312e378a79" (UID: "8881e503-831c-4519-b1b1-75312e378a79"). InnerVolumeSpecName "kube-api-access-x959x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:30:51.843816 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.843781    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x959x\" (UniqueName: \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-kube-api-access-x959x\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:51.843816 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.843811    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8881e503-831c-4519-b1b1-75312e378a79-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:51.843816 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.843821    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8881e503-831c-4519-b1b1-75312e378a79-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:51.844058 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:51.843830    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8881e503-831c-4519-b1b1-75312e378a79-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:30:52.444915 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.444878    5067 generic.go:356] "Generic (PLEG): container finished" podID="8881e503-831c-4519-b1b1-75312e378a79" containerID="fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4" exitCode=0
Nov 02 22:30:52.445139 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.444926    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"
Nov 02 22:30:52.445139 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.444961    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" event={"ID":"8881e503-831c-4519-b1b1-75312e378a79","Type":"ContainerDied","Data":"fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4"}
Nov 02 22:30:52.445139 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.445029    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42" event={"ID":"8881e503-831c-4519-b1b1-75312e378a79","Type":"ContainerDied","Data":"1c5c859b792be5da9fb52c45fea6a4ccb7eb5fc846b5c30ffc8c8b4414261a81"}
Nov 02 22:30:52.445139 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.445069    5067 scope.go:122] "RemoveContainer" containerID="fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4"
Nov 02 22:30:52.457642 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.457622    5067 scope.go:122] "RemoveContainer" containerID="e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047"
Nov 02 22:30:52.470504 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.469430    5067 scope.go:122] "RemoveContainer" containerID="fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4"
Nov 02 22:30:52.472940 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:52.472869    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4\": container with ID starting with fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4 not found: ID does not exist" containerID="fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4"
Nov 02 22:30:52.472940 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.472907    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4"} err="failed to get container status \"fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4\": rpc error: code = NotFound desc = could not find container \"fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4\": container with ID starting with fb001beb9a9cdc521c3744e9eaaac25557f5d5245150ee680a189feea74ca7b4 not found: ID does not exist"
Nov 02 22:30:52.472940 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.472932    5067 scope.go:122] "RemoveContainer" containerID="e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047"
Nov 02 22:30:52.473308 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:30:52.473289    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047\": container with ID starting with e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047 not found: ID does not exist" containerID="e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047"
Nov 02 22:30:52.473413 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.473313    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047"} err="failed to get container status \"e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047\": rpc error: code = NotFound desc = could not find container \"e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047\": container with ID starting with e7600ad5d95ab8a22d9fd02f8ff058ee955c89a2df99074c901842170c40f047 not found: ID does not exist"
Nov 02 22:30:52.648700 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.648668    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"]
Nov 02 22:30:52.694681 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:52.694650    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2e86dfb1-openshift-default-676cb89bd9-xdc42"]
Nov 02 22:30:53.847300 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:30:53.847264    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8881e503-831c-4519-b1b1-75312e378a79" path="/var/lib/kubelet/pods/8881e503-831c-4519-b1b1-75312e378a79/volumes"
Nov 02 22:31:07.578264 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:31:07.578208    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:31:37.577116 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:31:37.577078    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:31:50.975579 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:50.975531    5067 patch_prober.go:29] interesting pod/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:31:51.012073 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:50.975602    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:31:51.012073 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:50.997300    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"]
Nov 02 22:31:51.794408 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:51.794281    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"]
Nov 02 22:31:52.000018 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:51.999983    5067 generic.go:356] "Generic (PLEG): container finished" podID="95b6e129-f57a-4705-915b-52e87773e831" containerID="42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880" exitCode=0
Nov 02 22:31:52.001637 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.000060    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" event={"ID":"95b6e129-f57a-4705-915b-52e87773e831","Type":"ContainerDied","Data":"42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880"}
Nov 02 22:31:52.001637 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.000669    5067 scope.go:122] "RemoveContainer" containerID="42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880"
Nov 02 22:31:52.002936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.002708    5067 generic.go:356] "Generic (PLEG): container finished" podID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerID="aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6" exitCode=0
Nov 02 22:31:52.002936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.002776    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" event={"ID":"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01","Type":"ContainerDied","Data":"aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6"}
Nov 02 22:31:52.002936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.002906    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:31:52.003109 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.002975    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:31:52.003109 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.003010    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:31:52.003217 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.003160    5067 scope.go:122] "RemoveContainer" containerID="aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6"
Nov 02 22:31:52.004817 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.004791    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:31:52.004929 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.004863    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:31:52.004999 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:52.004939    5067 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 02 22:31:53.011764 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:53.011725    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" event={"ID":"95b6e129-f57a-4705-915b-52e87773e831","Type":"ContainerStarted","Data":"37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac"}
Nov 02 22:31:53.013969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:53.013912    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" event={"ID":"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01","Type":"ContainerStarted","Data":"67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f"}
Nov 02 22:31:53.655759 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:53.655727    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:31:53.656161 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:53.656062    5067 patch_prober.go:29] interesting pod/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 02 22:31:53.656354 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:53.656329    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 02 22:31:53.762832 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:53.762795    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:31:53.763197 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:53.763167    5067 patch_prober.go:29] interesting pod/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:31:53.763358 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:53.763332    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 02 22:31:54.658800 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:54.658756    5067 patch_prober.go:29] interesting pod/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 02 22:31:54.659311 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:54.658821    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 02 22:31:54.765390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:54.765123    5067 patch_prober.go:29] interesting pod/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:31:54.765390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:54.765193    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 02 22:31:55.657745 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:55.657706    5067 patch_prober.go:29] interesting pod/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 02 22:31:55.658045 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:55.658018    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 02 22:31:55.768063 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:55.768025    5067 patch_prober.go:29] interesting pod/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:31:55.768659 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:55.768630    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 02 22:31:56.655054 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:56.655014    5067 patch_prober.go:29] interesting pod/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 02 22:31:56.655357 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:56.655328    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 02 22:31:56.763131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:56.763090    5067 patch_prober.go:29] interesting pod/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:31:56.763422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:56.763393    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 02 22:31:57.654875 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:57.654831    5067 patch_prober.go:29] interesting pod/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 02 22:31:57.656248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:57.656213    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 02 22:31:57.775390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:57.765747    5067 patch_prober.go:29] interesting pod/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 02 22:31:57.775390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:57.765812    5067 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 02 22:31:58.306344 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:58.306305    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"]
Nov 02 22:31:58.660589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:58.660505    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:31:58.661201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:58.661171    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:31:58.661784 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:58.661765    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:31:58.769057 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:58.769015    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:31:58.770359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:58.769718    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:31:58.771569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:31:58.771545    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:32:01.693787 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:01.693745    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"]
Nov 02 22:32:03.307217 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:03.307112    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh" podUID="782626e4-258f-4569-bd26-9741d46f7909" containerName="payload-pre-processing" containerID="cri-o://9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b" gracePeriod=25
Nov 02 22:32:03.721942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:03.721909    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:32:03.790346 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:03.790305    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/782626e4-258f-4569-bd26-9741d46f7909-kube-api-access-zz8cr\" (UniqueName: \"kubernetes.io/projected/782626e4-258f-4569-bd26-9741d46f7909-kube-api-access-zz8cr\") pod \"782626e4-258f-4569-bd26-9741d46f7909\" (UID: \"782626e4-258f-4569-bd26-9741d46f7909\") "
Nov 02 22:32:03.790557 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:03.790421    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/782626e4-258f-4569-bd26-9741d46f7909-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/782626e4-258f-4569-bd26-9741d46f7909-plugins-config-volume\") pod \"782626e4-258f-4569-bd26-9741d46f7909\" (UID: \"782626e4-258f-4569-bd26-9741d46f7909\") "
Nov 02 22:32:03.790848 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:03.790822    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/782626e4-258f-4569-bd26-9741d46f7909-plugins-config-volume" pod "782626e4-258f-4569-bd26-9741d46f7909" (UID: "782626e4-258f-4569-bd26-9741d46f7909"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:32:03.791238 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:03.791218    5067 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/782626e4-258f-4569-bd26-9741d46f7909-plugins-config-volume\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:03.792933 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:03.792908    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/782626e4-258f-4569-bd26-9741d46f7909-kube-api-access-zz8cr" pod "782626e4-258f-4569-bd26-9741d46f7909" (UID: "782626e4-258f-4569-bd26-9741d46f7909"). InnerVolumeSpecName "kube-api-access-zz8cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:03.892301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:03.892267    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zz8cr\" (UniqueName: \"kubernetes.io/projected/782626e4-258f-4569-bd26-9741d46f7909-kube-api-access-zz8cr\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:04.183358 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:04.183325    5067 generic.go:356] "Generic (PLEG): container finished" podID="782626e4-258f-4569-bd26-9741d46f7909" containerID="9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b" exitCode=0
Nov 02 22:32:04.183547 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:04.183379    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"
Nov 02 22:32:04.183547 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:04.183408    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh" event={"ID":"782626e4-258f-4569-bd26-9741d46f7909","Type":"ContainerDied","Data":"9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b"}
Nov 02 22:32:04.183547 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:04.183458    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh" event={"ID":"782626e4-258f-4569-bd26-9741d46f7909","Type":"ContainerDied","Data":"436d75ebba5254cf18fd104ac6d87ec4fd43c478aca2cbdb339718818eddbf8e"}
Nov 02 22:32:04.183547 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:04.183481    5067 scope.go:122] "RemoveContainer" containerID="9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b"
Nov 02 22:32:04.196428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:04.196407    5067 scope.go:122] "RemoveContainer" containerID="9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b"
Nov 02 22:32:04.196716 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:04.196693    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b\": container with ID starting with 9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b not found: ID does not exist" containerID="9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b"
Nov 02 22:32:04.196764 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:04.196725    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b"} err="failed to get container status \"9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b\": rpc error: code = NotFound desc = could not find container \"9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b\": container with ID starting with 9b5881b25a07388489d392b59abf66dc3866df26cb56b9b73050b82698f3d70b not found: ID does not exist"
Nov 02 22:32:04.342344 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:04.342312    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"]
Nov 02 22:32:04.388475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:04.388432    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-ca85db-9f9d9bd99-qnfqh"]
Nov 02 22:32:05.847462 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:05.847426    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="782626e4-258f-4569-bd26-9741d46f7909" path="/var/lib/kubelet/pods/782626e4-258f-4569-bd26-9741d46f7909/volumes"
Nov 02 22:32:06.021360 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.021244    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" containerID="cri-o://381e0660c4513ad43df7a78785b077238642b1ce160136d815512828126490e4" gracePeriod=45
Nov 02 22:32:06.157428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.157385    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": dial tcp 10.132.0.56:8000: connect: connection refused"
Nov 02 22:32:06.217122 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.210852    5067 generic.go:356] "Generic (PLEG): container finished" podID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerID="381e0660c4513ad43df7a78785b077238642b1ce160136d815512828126490e4" exitCode=0
Nov 02 22:32:06.217122 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.210954    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" event={"ID":"b16554d0-6330-4ffe-89a4-5c91509a3f0e","Type":"ContainerDied","Data":"381e0660c4513ad43df7a78785b077238642b1ce160136d815512828126490e4"}
Nov 02 22:32:06.304318 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.304291    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:32:06.432560 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.432477    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b16554d0-6330-4ffe-89a4-5c91509a3f0e-kube-api-access-tq4dw\" (UniqueName: \"kubernetes.io/projected/b16554d0-6330-4ffe-89a4-5c91509a3f0e-kube-api-access-tq4dw\") pod \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") "
Nov 02 22:32:06.432705 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.432602    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tls-certs\" (UniqueName: \"kubernetes.io/secret/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tls-certs\") pod \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") "
Nov 02 22:32:06.432804 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.432787    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-dshm\") pod \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") "
Nov 02 22:32:06.432904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.432888    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tmp-dir\") pod \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") "
Nov 02 22:32:06.432962 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.432943    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-home\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-home\") pod \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") "
Nov 02 22:32:06.433026 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.433000    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-model-cache\") pod \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\" (UID: \"b16554d0-6330-4ffe-89a4-5c91509a3f0e\") "
Nov 02 22:32:06.433212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.433177    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tmp-dir" pod "b16554d0-6330-4ffe-89a4-5c91509a3f0e" (UID: "b16554d0-6330-4ffe-89a4-5c91509a3f0e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:06.433432 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.433408    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-home" pod "b16554d0-6330-4ffe-89a4-5c91509a3f0e" (UID: "b16554d0-6330-4ffe-89a4-5c91509a3f0e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:06.433615 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.433592    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-model-cache" pod "b16554d0-6330-4ffe-89a4-5c91509a3f0e" (UID: "b16554d0-6330-4ffe-89a4-5c91509a3f0e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:06.433809 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.433789    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:06.433882 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.433814    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:06.433882 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.433830    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:06.436260 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.436233    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tls-certs" pod "b16554d0-6330-4ffe-89a4-5c91509a3f0e" (UID: "b16554d0-6330-4ffe-89a4-5c91509a3f0e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:32:06.438787 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.438765    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b16554d0-6330-4ffe-89a4-5c91509a3f0e-kube-api-access-tq4dw" pod "b16554d0-6330-4ffe-89a4-5c91509a3f0e" (UID: "b16554d0-6330-4ffe-89a4-5c91509a3f0e"). InnerVolumeSpecName "kube-api-access-tq4dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:06.438945 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.438925    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-dshm" pod "b16554d0-6330-4ffe-89a4-5c91509a3f0e" (UID: "b16554d0-6330-4ffe-89a4-5c91509a3f0e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:06.535264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.535223    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq4dw\" (UniqueName: \"kubernetes.io/projected/b16554d0-6330-4ffe-89a4-5c91509a3f0e-kube-api-access-tq4dw\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:06.535264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.535266    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b16554d0-6330-4ffe-89a4-5c91509a3f0e-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:06.535524 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.535282    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b16554d0-6330-4ffe-89a4-5c91509a3f0e-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:06.694566 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.694482    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98" podUID="0a7d1d82-d0e6-450d-b160-8bb80e78fa94" containerName="payload-processing" containerID="cri-o://e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44" gracePeriod=25
Nov 02 22:32:06.816644 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:06.816599    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" containerID="cri-o://5b7840ca1281ba9d9b5e95c1486f2c48ffcc316be5eabb3f5e11eee4f5a91212" gracePeriod=45
Nov 02 22:32:07.253698 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.253428    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74" event={"ID":"b16554d0-6330-4ffe-89a4-5c91509a3f0e","Type":"ContainerDied","Data":"b5822346eebb673ab644f02d74aee7011ca2a4b426048e03458879286492d823"}
Nov 02 22:32:07.253698 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.253490    5067 scope.go:122] "RemoveContainer" containerID="381e0660c4513ad43df7a78785b077238642b1ce160136d815512828126490e4"
Nov 02 22:32:07.257530 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.254428    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"
Nov 02 22:32:07.263115 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.262970    5067 generic.go:356] "Generic (PLEG): container finished" podID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerID="5b7840ca1281ba9d9b5e95c1486f2c48ffcc316be5eabb3f5e11eee4f5a91212" exitCode=0
Nov 02 22:32:07.263115 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.263059    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" event={"ID":"2e148426-ee8e-477a-afcf-9e1674b4e056","Type":"ContainerDied","Data":"5b7840ca1281ba9d9b5e95c1486f2c48ffcc316be5eabb3f5e11eee4f5a91212"}
Nov 02 22:32:07.305255 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.305231    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:32:07.352604 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.352568    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-dshm\") pod \"2e148426-ee8e-477a-afcf-9e1674b4e056\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") "
Nov 02 22:32:07.352807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.352618    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-tmp-dir\") pod \"2e148426-ee8e-477a-afcf-9e1674b4e056\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") "
Nov 02 22:32:07.352807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.352778    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-model-cache\") pod \"2e148426-ee8e-477a-afcf-9e1674b4e056\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") "
Nov 02 22:32:07.352924 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.352895    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e148426-ee8e-477a-afcf-9e1674b4e056-kube-api-access-k7gt9\" (UniqueName: \"kubernetes.io/projected/2e148426-ee8e-477a-afcf-9e1674b4e056-kube-api-access-k7gt9\") pod \"2e148426-ee8e-477a-afcf-9e1674b4e056\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") "
Nov 02 22:32:07.353021 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.353006    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-home\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-home\") pod \"2e148426-ee8e-477a-afcf-9e1674b4e056\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") "
Nov 02 22:32:07.353133 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.353118    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2e148426-ee8e-477a-afcf-9e1674b4e056-tls-certs\" (UniqueName: \"kubernetes.io/secret/2e148426-ee8e-477a-afcf-9e1674b4e056-tls-certs\") pod \"2e148426-ee8e-477a-afcf-9e1674b4e056\" (UID: \"2e148426-ee8e-477a-afcf-9e1674b4e056\") "
Nov 02 22:32:07.353539 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.353509    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-model-cache" pod "2e148426-ee8e-477a-afcf-9e1674b4e056" (UID: "2e148426-ee8e-477a-afcf-9e1674b4e056"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:07.353927 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.353888    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-home" pod "2e148426-ee8e-477a-afcf-9e1674b4e056" (UID: "2e148426-ee8e-477a-afcf-9e1674b4e056"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:07.354817 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.354788    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-tmp-dir" pod "2e148426-ee8e-477a-afcf-9e1674b4e056" (UID: "2e148426-ee8e-477a-afcf-9e1674b4e056"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:07.354987 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.353873    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:07.361425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.355774    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-dshm" pod "2e148426-ee8e-477a-afcf-9e1674b4e056" (UID: "2e148426-ee8e-477a-afcf-9e1674b4e056"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:07.361425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.355867    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e148426-ee8e-477a-afcf-9e1674b4e056-tls-certs" pod "2e148426-ee8e-477a-afcf-9e1674b4e056" (UID: "2e148426-ee8e-477a-afcf-9e1674b4e056"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:32:07.370123 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.370089    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e148426-ee8e-477a-afcf-9e1674b4e056-kube-api-access-k7gt9" pod "2e148426-ee8e-477a-afcf-9e1674b4e056" (UID: "2e148426-ee8e-477a-afcf-9e1674b4e056"). InnerVolumeSpecName "kube-api-access-k7gt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:07.456465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.456279    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:07.456465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.456320    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e148426-ee8e-477a-afcf-9e1674b4e056-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:07.456465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.456336    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:07.456465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.456349    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e148426-ee8e-477a-afcf-9e1674b4e056-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:07.456465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.456405    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7gt9\" (UniqueName: \"kubernetes.io/projected/2e148426-ee8e-477a-afcf-9e1674b4e056-kube-api-access-k7gt9\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:07.542646 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.542558    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"]
Nov 02 22:32:07.549996 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.549962    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3224ea/simulator-w4-3224ea-kserve-7ccc744d68-85s74"]
Nov 02 22:32:07.595077 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:07.595041    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:32:07.869869 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:07.869790    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" path="/var/lib/kubelet/pods/b16554d0-6330-4ffe-89a4-5c91509a3f0e/volumes"
Nov 02 22:32:08.273125 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:08.273089    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" event={"ID":"2e148426-ee8e-477a-afcf-9e1674b4e056","Type":"ContainerDied","Data":"d25fbc0ebec596962b941e478f3d59d92451bd728ec9965a093edb85363d11d5"}
Nov 02 22:32:08.273621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:08.273129    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"
Nov 02 22:32:08.273621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:08.273134    5067 scope.go:122] "RemoveContainer" containerID="5b7840ca1281ba9d9b5e95c1486f2c48ffcc316be5eabb3f5e11eee4f5a91212"
Nov 02 22:32:08.282162 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:08.282128    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 02 22:32:08.311518 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:08.311470    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98" podUID="0a7d1d82-d0e6-450d-b160-8bb80e78fa94" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:32:08.430093 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:08.430047    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"]
Nov 02 22:32:08.439330 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:08.439287    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-ca85db/premium-w1-ca85db-kserve-f96fc4cc8-wskqj"]
Nov 02 22:32:09.849186 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:09.849147    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" path="/var/lib/kubelet/pods/2e148426-ee8e-477a-afcf-9e1674b4e056/volumes"
Nov 02 22:32:18.311039 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:18.311003    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98" podUID="0a7d1d82-d0e6-450d-b160-8bb80e78fa94" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 22:32:26.985086 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:26.985058    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:32:27.030491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.030454    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-kube-api-access-zm2f9\" (UniqueName: \"kubernetes.io/projected/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-kube-api-access-zm2f9\") pod \"0a7d1d82-d0e6-450d-b160-8bb80e78fa94\" (UID: \"0a7d1d82-d0e6-450d-b160-8bb80e78fa94\") "
Nov 02 22:32:27.030676 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.030497    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-plugins-config-volume\") pod \"0a7d1d82-d0e6-450d-b160-8bb80e78fa94\" (UID: \"0a7d1d82-d0e6-450d-b160-8bb80e78fa94\") "
Nov 02 22:32:27.030952 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.030929    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-plugins-config-volume" pod "0a7d1d82-d0e6-450d-b160-8bb80e78fa94" (UID: "0a7d1d82-d0e6-450d-b160-8bb80e78fa94"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:32:27.031255 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.031237    5067 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-plugins-config-volume\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:27.032715 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.032686    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-kube-api-access-zm2f9" pod "0a7d1d82-d0e6-450d-b160-8bb80e78fa94" (UID: "0a7d1d82-d0e6-450d-b160-8bb80e78fa94"). InnerVolumeSpecName "kube-api-access-zm2f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:27.133530 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.133493    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zm2f9\" (UniqueName: \"kubernetes.io/projected/0a7d1d82-d0e6-450d-b160-8bb80e78fa94-kube-api-access-zm2f9\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:27.417010 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.416968    5067 generic.go:356] "Generic (PLEG): container finished" podID="0a7d1d82-d0e6-450d-b160-8bb80e78fa94" containerID="e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44" exitCode=0
Nov 02 22:32:27.417208 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.417018    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"
Nov 02 22:32:27.417208 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.417053    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98" event={"ID":"0a7d1d82-d0e6-450d-b160-8bb80e78fa94","Type":"ContainerDied","Data":"e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44"}
Nov 02 22:32:27.417208 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.417113    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98" event={"ID":"0a7d1d82-d0e6-450d-b160-8bb80e78fa94","Type":"ContainerDied","Data":"6c1009138db9884e1a7123f8c66f9b20abdfc4547ea335776c2991c1e013dbb4"}
Nov 02 22:32:27.417208 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.417142    5067 scope.go:122] "RemoveContainer" containerID="e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44"
Nov 02 22:32:27.427931 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.427907    5067 scope.go:122] "RemoveContainer" containerID="e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44"
Nov 02 22:32:27.428256 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:27.428233    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44\": container with ID starting with e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44 not found: ID does not exist" containerID="e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44"
Nov 02 22:32:27.428347 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.428264    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44"} err="failed to get container status \"e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44\": rpc error: code = NotFound desc = could not find container \"e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44\": container with ID starting with e892172f2b5ce0d1ff6ba6b3b6c0121e514b011f6b88cc03a65fa01426806a44 not found: ID does not exist"
Nov 02 22:32:27.574778 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.574742    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"]
Nov 02 22:32:27.579105 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.579076    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3224ea-564895ddcc-bqg98"]
Nov 02 22:32:27.851464 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:27.851433    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a7d1d82-d0e6-450d-b160-8bb80e78fa94" path="/var/lib/kubelet/pods/0a7d1d82-d0e6-450d-b160-8bb80e78fa94/volumes"
Nov 02 22:32:37.579191 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:37.579153    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:32:40.688996 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:40.688959    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"]
Nov 02 22:32:40.737508 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:40.737468    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"]
Nov 02 22:32:45.081923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:45.081880    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"]
Nov 02 22:32:45.082438 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:45.082186    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy" containerID="cri-o://37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac" gracePeriod=30
Nov 02 22:32:46.883068 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:46.883023    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"]
Nov 02 22:32:46.927275 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:46.927234    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"]
Nov 02 22:32:46.930584 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:46.930555    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"]
Nov 02 22:32:49.649088 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:49.649048    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"]
Nov 02 22:32:49.649651 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:49.649409    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy" containerID="cri-o://ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01" gracePeriod=30
Nov 02 22:32:49.657488 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:49.657456    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"]
Nov 02 22:32:49.657765 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:49.657718    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy" containerID="cri-o://7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851" gracePeriod=30
Nov 02 22:32:49.663617 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:49.663592    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"]
Nov 02 22:32:49.663822 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:49.663797    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" containerID="cri-o://077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e" gracePeriod=30
Nov 02 22:32:49.844421 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:49.840762    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:32:49.844421 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:49.840840    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:32:49.844421 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:49.841207    5067 patch_prober.go:29] interesting pod/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:32:49.844421 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:49.841269    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:32:50.125870 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:50.125840    5067 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b6e129_f57a_4705_915b_52e87773e831.slice/crio-conmon-37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac.scope\": RecentStats: unable to find data in memory cache]"
Nov 02 22:32:50.365874 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.365812    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:32:50.410032 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.409997    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-envoy\") pod \"95b6e129-f57a-4705-915b-52e87773e831\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") "
Nov 02 22:32:50.410201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410044    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-certs\") pod \"95b6e129-f57a-4705-915b-52e87773e831\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") "
Nov 02 22:32:50.410201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410072    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-istio-token\" (UniqueName: \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-istio-token\") pod \"95b6e129-f57a-4705-915b-52e87773e831\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") "
Nov 02 22:32:50.410201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410107    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-socket\") pod \"95b6e129-f57a-4705-915b-52e87773e831\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") "
Nov 02 22:32:50.410201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410151    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-kube-api-access-hc4dj\" (UniqueName: \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-kube-api-access-hc4dj\") pod \"95b6e129-f57a-4705-915b-52e87773e831\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") "
Nov 02 22:32:50.410472 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410242    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/95b6e129-f57a-4705-915b-52e87773e831-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95b6e129-f57a-4705-915b-52e87773e831-istio-podinfo\") pod \"95b6e129-f57a-4705-915b-52e87773e831\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") "
Nov 02 22:32:50.410472 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410306    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-certs" pod "95b6e129-f57a-4705-915b-52e87773e831" (UID: "95b6e129-f57a-4705-915b-52e87773e831"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:50.410472 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410416    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-credential-socket\") pod \"95b6e129-f57a-4705-915b-52e87773e831\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") "
Nov 02 22:32:50.410625 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410473    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-data\") pod \"95b6e129-f57a-4705-915b-52e87773e831\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") "
Nov 02 22:32:50.410625 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410487    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-socket" pod "95b6e129-f57a-4705-915b-52e87773e831" (UID: "95b6e129-f57a-4705-915b-52e87773e831"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:50.410625 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410576    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95b6e129-f57a-4705-915b-52e87773e831-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95b6e129-f57a-4705-915b-52e87773e831-istiod-ca-cert\") pod \"95b6e129-f57a-4705-915b-52e87773e831\" (UID: \"95b6e129-f57a-4705-915b-52e87773e831\") "
Nov 02 22:32:50.410833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410670    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-credential-socket" pod "95b6e129-f57a-4705-915b-52e87773e831" (UID: "95b6e129-f57a-4705-915b-52e87773e831"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:50.410833 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.410761    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-data" pod "95b6e129-f57a-4705-915b-52e87773e831" (UID: "95b6e129-f57a-4705-915b-52e87773e831"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:50.411088 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.411066    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95b6e129-f57a-4705-915b-52e87773e831-istiod-ca-cert" pod "95b6e129-f57a-4705-915b-52e87773e831" (UID: "95b6e129-f57a-4705-915b-52e87773e831"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:32:50.411359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.411341    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:50.411434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.411403    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:50.411434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.411421    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:50.411508 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.411436    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:50.411508 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.411451    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95b6e129-f57a-4705-915b-52e87773e831-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:50.412625 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.412598    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-kube-api-access-hc4dj" pod "95b6e129-f57a-4705-915b-52e87773e831" (UID: "95b6e129-f57a-4705-915b-52e87773e831"). InnerVolumeSpecName "kube-api-access-hc4dj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:50.412719 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.412634    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-istio-token" pod "95b6e129-f57a-4705-915b-52e87773e831" (UID: "95b6e129-f57a-4705-915b-52e87773e831"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:50.412786 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.412746    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95b6e129-f57a-4705-915b-52e87773e831-istio-podinfo" pod "95b6e129-f57a-4705-915b-52e87773e831" (UID: "95b6e129-f57a-4705-915b-52e87773e831"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:32:50.412786 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.412772    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-envoy" pod "95b6e129-f57a-4705-915b-52e87773e831" (UID: "95b6e129-f57a-4705-915b-52e87773e831"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:50.512959 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.512911    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95b6e129-f57a-4705-915b-52e87773e831-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:50.512959 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.512944    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:50.512959 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.512960    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc4dj\" (UniqueName: \"kubernetes.io/projected/95b6e129-f57a-4705-915b-52e87773e831-kube-api-access-hc4dj\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:50.513287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.512972    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95b6e129-f57a-4705-915b-52e87773e831-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:50.663012 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.662919    5067 generic.go:356] "Generic (PLEG): container finished" podID="95b6e129-f57a-4705-915b-52e87773e831" containerID="37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac" exitCode=0
Nov 02 22:32:50.663012 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.662983    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"
Nov 02 22:32:50.663012 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.662999    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" event={"ID":"95b6e129-f57a-4705-915b-52e87773e831","Type":"ContainerDied","Data":"37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac"}
Nov 02 22:32:50.663641 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.663054    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6" event={"ID":"95b6e129-f57a-4705-915b-52e87773e831","Type":"ContainerDied","Data":"e8a6df517dea6ba58323a980c32a56f3600782bad3dcf8c7de7e7dbc9abb29ec"}
Nov 02 22:32:50.663641 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.663077    5067 scope.go:122] "RemoveContainer" containerID="37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac"
Nov 02 22:32:50.674255 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.674236    5067 scope.go:122] "RemoveContainer" containerID="42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880"
Nov 02 22:32:50.683229 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.683209    5067 scope.go:122] "RemoveContainer" containerID="37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac"
Nov 02 22:32:50.683519 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:50.683499    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac\": container with ID starting with 37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac not found: ID does not exist" containerID="37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac"
Nov 02 22:32:50.683587 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.683527    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac"} err="failed to get container status \"37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac\": rpc error: code = NotFound desc = could not find container \"37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac\": container with ID starting with 37531c00acdb10cfc4983bfbe8d7240f6965bad1e9a83cc1a95fc2f862e3d4ac not found: ID does not exist"
Nov 02 22:32:50.683587 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.683545    5067 scope.go:122] "RemoveContainer" containerID="42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880"
Nov 02 22:32:50.683770 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:50.683755    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880\": container with ID starting with 42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880 not found: ID does not exist" containerID="42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880"
Nov 02 22:32:50.683806 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.683775    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880"} err="failed to get container status \"42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880\": rpc error: code = NotFound desc = could not find container \"42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880\": container with ID starting with 42a3990310e2e19227e988837991a73d9dff0eda04a0cbbc0a5a621a59ca5880 not found: ID does not exist"
Nov 02 22:32:50.829031 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.828997    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"]
Nov 02 22:32:50.834956 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:50.834929    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fb670683-openshift-default-67d689b6df-rjqg6"]
Nov 02 22:32:51.846346 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:51.846312    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95b6e129-f57a-4705-915b-52e87773e831" path="/var/lib/kubelet/pods/95b6e129-f57a-4705-915b-52e87773e831/volumes"
Nov 02 22:32:55.033417 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.033385    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:32:55.086413 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.082212    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:32:55.086948 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.086919    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194521    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-token\" (UniqueName: \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-token\") pod \"017257d4-2273-4bd7-a759-f7537dfda7c5\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194575    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-data\") pod \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194623    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-data\") pod \"017257d4-2273-4bd7-a759-f7537dfda7c5\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194667    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-envoy\") pod \"017257d4-2273-4bd7-a759-f7537dfda7c5\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194697    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-credential-socket\") pod \"17acd4de-db2e-4d59-a748-a78da66499c9\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194730    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-credential-socket\") pod \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194780    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-kube-api-access-m9fp7\" (UniqueName: \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-kube-api-access-m9fp7\") pod \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194811    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-kube-api-access-slscm\" (UniqueName: \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-kube-api-access-slscm\") pod \"017257d4-2273-4bd7-a759-f7537dfda7c5\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194842    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/17acd4de-db2e-4d59-a748-a78da66499c9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17acd4de-db2e-4d59-a748-a78da66499c9-istio-podinfo\") pod \"17acd4de-db2e-4d59-a748-a78da66499c9\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194908    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-token\" (UniqueName: \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-token\") pod \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194938    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-podinfo\") pod \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.194969    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-socket\") pod \"17acd4de-db2e-4d59-a748-a78da66499c9\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.195009    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-certs\") pod \"017257d4-2273-4bd7-a759-f7537dfda7c5\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.195060    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istiod-ca-cert\") pod \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.195103    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-envoy\") pod \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.195168    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-socket\") pod \"017257d4-2273-4bd7-a759-f7537dfda7c5\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.195201    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-certs\") pod \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.195256    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-data\") pod \"17acd4de-db2e-4d59-a748-a78da66499c9\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.195299    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-certs\") pod \"17acd4de-db2e-4d59-a748-a78da66499c9\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196093    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-envoy\") pod \"17acd4de-db2e-4d59-a748-a78da66499c9\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196149    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17acd4de-db2e-4d59-a748-a78da66499c9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17acd4de-db2e-4d59-a748-a78da66499c9-istiod-ca-cert\") pod \"17acd4de-db2e-4d59-a748-a78da66499c9\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196257    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-credential-socket\") pod \"017257d4-2273-4bd7-a759-f7537dfda7c5\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196327    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-podinfo\") pod \"017257d4-2273-4bd7-a759-f7537dfda7c5\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196386    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-data" pod "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" (UID: "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196433    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-socket\") pod \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\" (UID: \"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196490    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-istio-token\" (UniqueName: \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-istio-token\") pod \"17acd4de-db2e-4d59-a748-a78da66499c9\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196551    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-kube-api-access-2mrzb\" (UniqueName: \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-kube-api-access-2mrzb\") pod \"17acd4de-db2e-4d59-a748-a78da66499c9\" (UID: \"17acd4de-db2e-4d59-a748-a78da66499c9\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196604    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/017257d4-2273-4bd7-a759-f7537dfda7c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/017257d4-2273-4bd7-a759-f7537dfda7c5-istiod-ca-cert\") pod \"017257d4-2273-4bd7-a759-f7537dfda7c5\" (UID: \"017257d4-2273-4bd7-a759-f7537dfda7c5\") "
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196652    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-certs" pod "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" (UID: "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196857    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-data" pod "017257d4-2273-4bd7-a759-f7537dfda7c5" (UID: "017257d4-2273-4bd7-a759-f7537dfda7c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.196980    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-credential-socket" pod "17acd4de-db2e-4d59-a748-a78da66499c9" (UID: "17acd4de-db2e-4d59-a748-a78da66499c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.197296    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/017257d4-2273-4bd7-a759-f7537dfda7c5-istiod-ca-cert" pod "017257d4-2273-4bd7-a759-f7537dfda7c5" (UID: "017257d4-2273-4bd7-a759-f7537dfda7c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.197408    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-socket" pod "017257d4-2273-4bd7-a759-f7537dfda7c5" (UID: "017257d4-2273-4bd7-a759-f7537dfda7c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.197589    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-data" pod "17acd4de-db2e-4d59-a748-a78da66499c9" (UID: "17acd4de-db2e-4d59-a748-a78da66499c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.197671    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/017257d4-2273-4bd7-a759-f7537dfda7c5-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.197693    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.197708    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.197722    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.197737    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.197751    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.197786    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-certs" pod "17acd4de-db2e-4d59-a748-a78da66499c9" (UID: "17acd4de-db2e-4d59-a748-a78da66499c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.198033    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-socket" pod "17acd4de-db2e-4d59-a748-a78da66499c9" (UID: "17acd4de-db2e-4d59-a748-a78da66499c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.208707 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.198313    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-certs" pod "017257d4-2273-4bd7-a759-f7537dfda7c5" (UID: "017257d4-2273-4bd7-a759-f7537dfda7c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.211927 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.211850    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-credential-socket" pod "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" (UID: "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.212108 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.212061    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istiod-ca-cert" pod "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" (UID: "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:32:55.212285 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.212256    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-socket" pod "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" (UID: "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.212468 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.212314    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17acd4de-db2e-4d59-a748-a78da66499c9-istiod-ca-cert" pod "17acd4de-db2e-4d59-a748-a78da66499c9" (UID: "17acd4de-db2e-4d59-a748-a78da66499c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:32:55.212602 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.212472    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-credential-socket" pod "017257d4-2273-4bd7-a759-f7537dfda7c5" (UID: "017257d4-2273-4bd7-a759-f7537dfda7c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.216463 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.216435    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-kube-api-access-2mrzb" pod "17acd4de-db2e-4d59-a748-a78da66499c9" (UID: "17acd4de-db2e-4d59-a748-a78da66499c9"). InnerVolumeSpecName "kube-api-access-2mrzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:55.219110 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.219016    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17acd4de-db2e-4d59-a748-a78da66499c9-istio-podinfo" pod "17acd4de-db2e-4d59-a748-a78da66499c9" (UID: "17acd4de-db2e-4d59-a748-a78da66499c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:32:55.219110 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.219062    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-podinfo" pod "017257d4-2273-4bd7-a759-f7537dfda7c5" (UID: "017257d4-2273-4bd7-a759-f7537dfda7c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:32:55.219110 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.219067    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-kube-api-access-slscm" pod "017257d4-2273-4bd7-a759-f7537dfda7c5" (UID: "017257d4-2273-4bd7-a759-f7537dfda7c5"). InnerVolumeSpecName "kube-api-access-slscm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:55.219328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.219199    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-envoy" pod "017257d4-2273-4bd7-a759-f7537dfda7c5" (UID: "017257d4-2273-4bd7-a759-f7537dfda7c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.219328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.219231    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-token" pod "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" (UID: "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:55.219659 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.219582    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-token" pod "017257d4-2273-4bd7-a759-f7537dfda7c5" (UID: "017257d4-2273-4bd7-a759-f7537dfda7c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:55.219659 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.219587    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-kube-api-access-m9fp7" pod "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" (UID: "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97"). InnerVolumeSpecName "kube-api-access-m9fp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:55.219659 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.219600    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-envoy" pod "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" (UID: "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.219659 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.219638    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-envoy" pod "17acd4de-db2e-4d59-a748-a78da66499c9" (UID: "17acd4de-db2e-4d59-a748-a78da66499c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:55.220305 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.220279    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-podinfo" pod "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" (UID: "e5ac378c-d84e-4bc1-9fd6-595af7ad3f97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:32:55.220733 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.220708    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-istio-token" pod "17acd4de-db2e-4d59-a748-a78da66499c9" (UID: "17acd4de-db2e-4d59-a748-a78da66499c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.299900    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.299940    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.299956    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9fp7\" (UniqueName: \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-kube-api-access-m9fp7\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.299973    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slscm\" (UniqueName: \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-kube-api-access-slscm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300023    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17acd4de-db2e-4d59-a748-a78da66499c9-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300039    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300054    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300067    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300081    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300095    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300112    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300125    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300140    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300152    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17acd4de-db2e-4d59-a748-a78da66499c9-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300166    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17acd4de-db2e-4d59-a748-a78da66499c9-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300180    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/017257d4-2273-4bd7-a759-f7537dfda7c5-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300195    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300208    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300222    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300235    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mrzb\" (UniqueName: \"kubernetes.io/projected/17acd4de-db2e-4d59-a748-a78da66499c9-kube-api-access-2mrzb\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.300276 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.300249    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/017257d4-2273-4bd7-a759-f7537dfda7c5-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:55.706713 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.706682    5067 generic.go:356] "Generic (PLEG): container finished" podID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerID="7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851" exitCode=0
Nov 02 22:32:55.706882 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.706749    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"
Nov 02 22:32:55.706882 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.706768    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" event={"ID":"017257d4-2273-4bd7-a759-f7537dfda7c5","Type":"ContainerDied","Data":"7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851"}
Nov 02 22:32:55.706882 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.706843    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx" event={"ID":"017257d4-2273-4bd7-a759-f7537dfda7c5","Type":"ContainerDied","Data":"799243be8f9cee76eb92c6e83aecb2478bc3ce4103811a7a184db235190bae0f"}
Nov 02 22:32:55.706882 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.706868    5067 scope.go:122] "RemoveContainer" containerID="7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851"
Nov 02 22:32:55.708566 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.708543    5067 generic.go:356] "Generic (PLEG): container finished" podID="17acd4de-db2e-4d59-a748-a78da66499c9" containerID="ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01" exitCode=0
Nov 02 22:32:55.708755 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.708612    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"
Nov 02 22:32:55.708846 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.708629    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" event={"ID":"17acd4de-db2e-4d59-a748-a78da66499c9","Type":"ContainerDied","Data":"ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01"}
Nov 02 22:32:55.708910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.708871    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc" event={"ID":"17acd4de-db2e-4d59-a748-a78da66499c9","Type":"ContainerDied","Data":"dd8cb12e3eac46b4399d28e05d1939e212c595f19b3bd78f1d6f4116d23db69e"}
Nov 02 22:32:55.710406 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.710377    5067 generic.go:356] "Generic (PLEG): container finished" podID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerID="077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e" exitCode=0
Nov 02 22:32:55.710507 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.710426    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" event={"ID":"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97","Type":"ContainerDied","Data":"077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e"}
Nov 02 22:32:55.710507 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.710464    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" event={"ID":"e5ac378c-d84e-4bc1-9fd6-595af7ad3f97","Type":"ContainerDied","Data":"4f4f2b6468bf1e68723938eef62263d33e432e07ba91b12d4dd90ff7fade4295"}
Nov 02 22:32:55.710630 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.710581    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:32:55.713380 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.713242    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main" containerID="cri-o://5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54" gracePeriod=45
Nov 02 22:32:55.720040 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.719998    5067 scope.go:122] "RemoveContainer" containerID="7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851"
Nov 02 22:32:55.720307 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:55.720281    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851\": container with ID starting with 7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851 not found: ID does not exist" containerID="7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851"
Nov 02 22:32:55.720416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.720309    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851"} err="failed to get container status \"7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851\": rpc error: code = NotFound desc = could not find container \"7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851\": container with ID starting with 7d3a88177cd46cc25969c0017865c06f33eb629b493ffe9d4829d41419cef851 not found: ID does not exist"
Nov 02 22:32:55.720416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.720332    5067 scope.go:122] "RemoveContainer" containerID="ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01"
Nov 02 22:32:55.758727 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.758691    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" containerID="cri-o://26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0" gracePeriod=45
Nov 02 22:32:55.780716 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.780680    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": dial tcp 10.132.0.53:8000: connect: connection refused"
Nov 02 22:32:55.870002 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.869983    5067 scope.go:122] "RemoveContainer" containerID="ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01"
Nov 02 22:32:55.870398 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:55.870346    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01\": container with ID starting with ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01 not found: ID does not exist" containerID="ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01"
Nov 02 22:32:55.870458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.870393    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01"} err="failed to get container status \"ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01\": rpc error: code = NotFound desc = could not find container \"ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01\": container with ID starting with ff9aa9a18c19f3fa58fcc69bf54b37fadbf3e4404f5d5374384400dcb86c6b01 not found: ID does not exist"
Nov 02 22:32:55.870458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.870418    5067 scope.go:122] "RemoveContainer" containerID="077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e"
Nov 02 22:32:55.881884 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.881809    5067 scope.go:122] "RemoveContainer" containerID="077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e"
Nov 02 22:32:55.882141 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:55.882115    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e\": container with ID starting with 077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e not found: ID does not exist" containerID="077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e"
Nov 02 22:32:55.882225 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.882152    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e"} err="failed to get container status \"077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e\": rpc error: code = NotFound desc = could not find container \"077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e\": container with ID starting with 077fd8435e677353eeab7705e5c952a4394ec437c60603dea642f6d0fe7e698e not found: ID does not exist"
Nov 02 22:32:55.974434 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:55.974394    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"]
Nov 02 22:32:56.023045 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.023006    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-rkfqx"]
Nov 02 22:32:56.053125 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.053097    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:32:56.067288 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.067267    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:32:56.082631 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.082606    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"]
Nov 02 22:32:56.089337 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.089311    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-k4kxc"]
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.215500    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-tmp-dir\") pod \"28414ade-e581-4334-9bd9-0d85a4aa33c6\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.215574    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28414ade-e581-4334-9bd9-0d85a4aa33c6-tls-certs\" (UniqueName: \"kubernetes.io/secret/28414ade-e581-4334-9bd9-0d85a4aa33c6-tls-certs\") pod \"28414ade-e581-4334-9bd9-0d85a4aa33c6\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.215606    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-dshm\") pod \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.215642    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-dshm\") pod \"28414ade-e581-4334-9bd9-0d85a4aa33c6\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.215686    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-home\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-home\") pod \"28414ade-e581-4334-9bd9-0d85a4aa33c6\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.215768    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28414ade-e581-4334-9bd9-0d85a4aa33c6-kube-api-access-g7zr7\" (UniqueName: \"kubernetes.io/projected/28414ade-e581-4334-9bd9-0d85a4aa33c6-kube-api-access-g7zr7\") pod \"28414ade-e581-4334-9bd9-0d85a4aa33c6\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.215893    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-model-cache\") pod \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.215929    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tmp-dir\") pod \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.215990    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-model-cache\") pod \"28414ade-e581-4334-9bd9-0d85a4aa33c6\" (UID: \"28414ade-e581-4334-9bd9-0d85a4aa33c6\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.216028    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-kube-api-access-l5mds\" (UniqueName: \"kubernetes.io/projected/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-kube-api-access-l5mds\") pod \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.216076    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tls-certs\") pod \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.216113    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-home\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-home\") pod \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\" (UID: \"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0\") "
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.223531    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-home" pod "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" (UID: "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.223721    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-model-cache" pod "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" (UID: "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.223795    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-tmp-dir" pod "28414ade-e581-4334-9bd9-0d85a4aa33c6" (UID: "28414ade-e581-4334-9bd9-0d85a4aa33c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.223911    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tmp-dir" pod "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" (UID: "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:56.225113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.224085    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-model-cache" pod "28414ade-e581-4334-9bd9-0d85a4aa33c6" (UID: "28414ade-e581-4334-9bd9-0d85a4aa33c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:56.229290 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.228791    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-home" pod "28414ade-e581-4334-9bd9-0d85a4aa33c6" (UID: "28414ade-e581-4334-9bd9-0d85a4aa33c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:56.251918 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.251841    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-dshm" pod "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" (UID: "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:56.252314 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.252207    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-dshm" pod "28414ade-e581-4334-9bd9-0d85a4aa33c6" (UID: "28414ade-e581-4334-9bd9-0d85a4aa33c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:32:56.281386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.275555    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-kube-api-access-l5mds" pod "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" (UID: "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0"). InnerVolumeSpecName "kube-api-access-l5mds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:56.281386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.275647    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tls-certs" pod "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" (UID: "a0eb1db6-6b93-4b5b-be68-ea87fcead2f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:32:56.281386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.277415    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28414ade-e581-4334-9bd9-0d85a4aa33c6-tls-certs" pod "28414ade-e581-4334-9bd9-0d85a4aa33c6" (UID: "28414ade-e581-4334-9bd9-0d85a4aa33c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:32:56.287442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.285347    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28414ade-e581-4334-9bd9-0d85a4aa33c6-kube-api-access-g7zr7" pod "28414ade-e581-4334-9bd9-0d85a4aa33c6" (UID: "28414ade-e581-4334-9bd9-0d85a4aa33c6"). InnerVolumeSpecName "kube-api-access-g7zr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:32:56.335304 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335266    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28414ade-e581-4334-9bd9-0d85a4aa33c6-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335304 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335305    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335321    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335335    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335348    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7zr7\" (UniqueName: \"kubernetes.io/projected/28414ade-e581-4334-9bd9-0d85a4aa33c6-kube-api-access-g7zr7\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335378    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335392    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335408    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335423    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5mds\" (UniqueName: \"kubernetes.io/projected/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-kube-api-access-l5mds\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335437    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335450    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.335634 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.335462    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28414ade-e581-4334-9bd9-0d85a4aa33c6-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:32:56.729130 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.729095    5067 generic.go:356] "Generic (PLEG): container finished" podID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerID="26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0" exitCode=0
Nov 02 22:32:56.729359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.729153    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"
Nov 02 22:32:56.729359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.729182    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" event={"ID":"28414ade-e581-4334-9bd9-0d85a4aa33c6","Type":"ContainerDied","Data":"26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0"}
Nov 02 22:32:56.729359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.729232    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh" event={"ID":"28414ade-e581-4334-9bd9-0d85a4aa33c6","Type":"ContainerDied","Data":"eacbb0e32f4395257f27c6180e61e434eb91587a1fe45c4a750655a10cf2c832"}
Nov 02 22:32:56.729359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.729253    5067 scope.go:122] "RemoveContainer" containerID="26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0"
Nov 02 22:32:56.731115 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.731079    5067 generic.go:356] "Generic (PLEG): container finished" podID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerID="5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54" exitCode=0
Nov 02 22:32:56.731242 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.731110    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" event={"ID":"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0","Type":"ContainerDied","Data":"5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54"}
Nov 02 22:32:56.731242 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.731138    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"
Nov 02 22:32:56.731242 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.731156    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h" event={"ID":"a0eb1db6-6b93-4b5b-be68-ea87fcead2f0","Type":"ContainerDied","Data":"27ca9edcb7ecc488f1c9d7b060085e2d7a498a6d99fc7c19bd384ebfe772c7ca"}
Nov 02 22:32:56.740914 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.740893    5067 scope.go:122] "RemoveContainer" containerID="26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0"
Nov 02 22:32:56.741252 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:56.741221    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0\": container with ID starting with 26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0 not found: ID does not exist" containerID="26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0"
Nov 02 22:32:56.741334 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.741262    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0"} err="failed to get container status \"26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0\": rpc error: code = NotFound desc = could not find container \"26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0\": container with ID starting with 26ad4185e02278a41e387b721166b85f3edbc09ab9098716db3c841499e7bbb0 not found: ID does not exist"
Nov 02 22:32:56.741334 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.741288    5067 scope.go:122] "RemoveContainer" containerID="5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54"
Nov 02 22:32:56.750240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.750223    5067 scope.go:122] "RemoveContainer" containerID="5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54"
Nov 02 22:32:56.750508 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:32:56.750489    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54\": container with ID starting with 5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54 not found: ID does not exist" containerID="5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54"
Nov 02 22:32:56.750553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.750514    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54"} err="failed to get container status \"5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54\": rpc error: code = NotFound desc = could not find container \"5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54\": container with ID starting with 5a920228c5e04750bf048bf7f062c3c1aa299819f824616c6d4dcc316dc3fc54 not found: ID does not exist"
Nov 02 22:32:56.893864 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.893821    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"]
Nov 02 22:32:56.897488 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.897461    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-49a557/simulator-w6-49a557-kserve-67589c998-zf4dh"]
Nov 02 22:32:56.999265 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:56.999230    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"]
Nov 02 22:32:57.005258 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:57.005232    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-49a557/distinct-2-w6-49a557-kserve-c986f8757-rtp4h"]
Nov 02 22:32:57.849042 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:57.849005    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" path="/var/lib/kubelet/pods/017257d4-2273-4bd7-a759-f7537dfda7c5/volumes"
Nov 02 22:32:57.849569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:57.849554    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" path="/var/lib/kubelet/pods/17acd4de-db2e-4d59-a748-a78da66499c9/volumes"
Nov 02 22:32:57.849980 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:57.849967    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" path="/var/lib/kubelet/pods/28414ade-e581-4334-9bd9-0d85a4aa33c6/volumes"
Nov 02 22:32:57.850723 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:57.850707    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" path="/var/lib/kubelet/pods/a0eb1db6-6b93-4b5b-be68-ea87fcead2f0/volumes"
Nov 02 22:32:59.646283 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646243    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"]
Nov 02 22:32:59.646932 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646913    5067 cpu_manager.go:414] "containerMap: removing container" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main"
Nov 02 22:32:59.646992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646933    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main"
Nov 02 22:32:59.646992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646950    5067 cpu_manager.go:414] "containerMap: removing container" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main"
Nov 02 22:32:59.646992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646955    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main"
Nov 02 22:32:59.646992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646967    5067 cpu_manager.go:414] "containerMap: removing container" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy"
Nov 02 22:32:59.646992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646972    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy"
Nov 02 22:32:59.646992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646981    5067 cpu_manager.go:414] "containerMap: removing container" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy"
Nov 02 22:32:59.646992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646986    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy"
Nov 02 22:32:59.646992 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646995    5067 cpu_manager.go:414] "containerMap: removing container" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.646999    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647009    5067 cpu_manager.go:414] "containerMap: removing container" podUID="782626e4-258f-4569-bd26-9741d46f7909" containerName="payload-pre-processing"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647013    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="782626e4-258f-4569-bd26-9741d46f7909" containerName="payload-pre-processing"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647021    5067 cpu_manager.go:414] "containerMap: removing container" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647025    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647038    5067 cpu_manager.go:414] "containerMap: removing container" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647042    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647049    5067 cpu_manager.go:414] "containerMap: removing container" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647053    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647064    5067 cpu_manager.go:414] "containerMap: removing container" podUID="0a7d1d82-d0e6-450d-b160-8bb80e78fa94" containerName="payload-processing"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647069    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a7d1d82-d0e6-450d-b160-8bb80e78fa94" containerName="payload-processing"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647078    5067 cpu_manager.go:414] "containerMap: removing container" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647083    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647095    5067 cpu_manager.go:414] "containerMap: removing container" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647099    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647178    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="782626e4-258f-4569-bd26-9741d46f7909" containerName="payload-pre-processing"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647186    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647192    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647197    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="28414ade-e581-4334-9bd9-0d85a4aa33c6" containerName="main"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647204    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a7d1d82-d0e6-450d-b160-8bb80e78fa94" containerName="payload-processing"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647210    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647215    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="17acd4de-db2e-4d59-a748-a78da66499c9" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647221    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="017257d4-2273-4bd7-a759-f7537dfda7c5" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647228    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0eb1db6-6b93-4b5b-be68-ea87fcead2f0" containerName="main"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647235    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="b16554d0-6330-4ffe-89a4-5c91509a3f0e" containerName="main"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647240    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy"
Nov 02 22:32:59.647240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.647245    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e148426-ee8e-477a-afcf-9e1674b4e056" containerName="main"
Nov 02 22:32:59.651164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.651137    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:32:59.697766 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.697734    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-xks-ca\") pod \"maas-api-revoke-sub-74121fcd-2c9sn\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:32:59.697939 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.697919    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-74121fcd-2c9sn\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:32:59.698104 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.698084    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpvt2\" (UniqueName: \"kubernetes.io/projected/5feca907-056a-416b-8cf7-246fc20b2194-kube-api-access-gpvt2\") pod \"maas-api-revoke-sub-74121fcd-2c9sn\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:32:59.703300 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.703217    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 22:32:59.707832 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.707807    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"]
Nov 02 22:32:59.749659 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.749623    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cndr2\""
Nov 02 22:32:59.799923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.799890    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-74121fcd-2c9sn\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:32:59.800093 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.799986    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpvt2\" (UniqueName: \"kubernetes.io/projected/5feca907-056a-416b-8cf7-246fc20b2194-kube-api-access-gpvt2\") pod \"maas-api-revoke-sub-74121fcd-2c9sn\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:32:59.800135 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.800117    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-xks-ca\") pod \"maas-api-revoke-sub-74121fcd-2c9sn\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:32:59.800658 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.800635    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-74121fcd-2c9sn\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:32:59.802591 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.802573    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-xks-ca\") pod \"maas-api-revoke-sub-74121fcd-2c9sn\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:32:59.857781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.857749    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpvt2\" (UniqueName: \"kubernetes.io/projected/5feca907-056a-416b-8cf7-246fc20b2194-kube-api-access-gpvt2\") pod \"maas-api-revoke-sub-74121fcd-2c9sn\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:32:59.878922 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.878891    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"]
Nov 02 22:32:59.879716 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.879694    5067 cpu_manager.go:414] "containerMap: removing container" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy"
Nov 02 22:32:59.879716 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.879716    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8881e503-831c-4519-b1b1-75312e378a79" containerName="istio-proxy"
Nov 02 22:32:59.879911 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.879841    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="95b6e129-f57a-4705-915b-52e87773e831" containerName="istio-proxy"
Nov 02 22:32:59.883034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.883012    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:32:59.901411 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.901308    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnm9q\" (UniqueName: \"kubernetes.io/projected/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-kube-api-access-pnm9q\") pod \"maas-api-revoke-sub-a4f91c03-qg2wb\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:32:59.901545 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.901482    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4f91c03-qg2wb\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:32:59.901655 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.901638    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4f91c03-qg2wb\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:32:59.938598 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.938118    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"]
Nov 02 22:32:59.968944 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:32:59.968847    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:33:00.003649 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.002794    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnm9q\" (UniqueName: \"kubernetes.io/projected/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-kube-api-access-pnm9q\") pod \"maas-api-revoke-sub-a4f91c03-qg2wb\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:33:00.003649 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.002883    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4f91c03-qg2wb\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:33:00.003649 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.003019    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4f91c03-qg2wb\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:33:00.005087 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.005059    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4f91c03-qg2wb\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:33:00.008218 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.008171    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4f91c03-qg2wb\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:33:00.063336 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.063300    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnm9q\" (UniqueName: \"kubernetes.io/projected/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-kube-api-access-pnm9q\") pod \"maas-api-revoke-sub-a4f91c03-qg2wb\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:33:00.204469 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.204376    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:33:00.416604 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.416556    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"]
Nov 02 22:33:00.639642 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.639492    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"]
Nov 02 22:33:00.646598 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:33:00.646572    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b0584fc_060b_4c59_a5ff_34a7ae51b1d0.slice/crio-305c198e6ec80dfa74f7bacda8f483cb8114ef2f2323aace769e2b126aad9f21 WatchSource:0}: Error finding container 305c198e6ec80dfa74f7bacda8f483cb8114ef2f2323aace769e2b126aad9f21: Status 404 returned error can't find the container with id 305c198e6ec80dfa74f7bacda8f483cb8114ef2f2323aace769e2b126aad9f21
Nov 02 22:33:00.772475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.772431    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb" event={"ID":"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0","Type":"ContainerStarted","Data":"305c198e6ec80dfa74f7bacda8f483cb8114ef2f2323aace769e2b126aad9f21"}
Nov 02 22:33:00.774083 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:00.774054    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn" event={"ID":"5feca907-056a-416b-8cf7-246fc20b2194","Type":"ContainerStarted","Data":"0e86d19e63dd39c3e91b3edde89e0db3b837ecab386d67a297478e3777b821a9"}
Nov 02 22:33:01.781575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:01.781542    5067 generic.go:356] "Generic (PLEG): container finished" podID="9b0584fc-060b-4c59-a5ff-34a7ae51b1d0" containerID="077c22cd5c3fee756c13bd53c58b8f40b8f1e8642c104293f4090fddd5dd173c" exitCode=0
Nov 02 22:33:01.782074 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:01.781630    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb" event={"ID":"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0","Type":"ContainerDied","Data":"077c22cd5c3fee756c13bd53c58b8f40b8f1e8642c104293f4090fddd5dd173c"}
Nov 02 22:33:01.783471 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:01.783442    5067 generic.go:356] "Generic (PLEG): container finished" podID="5feca907-056a-416b-8cf7-246fc20b2194" containerID="754d22b8d78d2fa14ee33207ea12addcd98b49b0d42ce5d0dcc34798b4c44388" exitCode=0
Nov 02 22:33:01.783588 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:01.783523    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn" event={"ID":"5feca907-056a-416b-8cf7-246fc20b2194","Type":"ContainerDied","Data":"754d22b8d78d2fa14ee33207ea12addcd98b49b0d42ce5d0dcc34798b4c44388"}
Nov 02 22:33:01.905606 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:01.905574    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main" containerID="cri-o://028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695" gracePeriod=45
Nov 02 22:33:01.950803 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:01.950763    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main" containerID="cri-o://aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39" gracePeriod=45
Nov 02 22:33:01.952166 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:01.952137    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" containerID="cri-o://636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23" gracePeriod=45
Nov 02 22:33:02.105665 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.105202    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": dial tcp 10.132.0.55:8000: connect: connection refused"
Nov 02 22:33:02.287543 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.287476    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:33:02.293824 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.293797    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:33:02.296518 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.296496    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:33:02.384788 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.384759    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-dshm\") pod \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") "
Nov 02 22:33:02.384964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.384835    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tmp-dir\") pod \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") "
Nov 02 22:33:02.384964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.384871    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tls-certs\" (UniqueName: \"kubernetes.io/secret/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tls-certs\") pod \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") "
Nov 02 22:33:02.384964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.384893    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-home\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-home\") pod \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") "
Nov 02 22:33:02.384964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.384922    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-dshm\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-dshm\") pod \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") "
Nov 02 22:33:02.385140 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385051    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-model-cache\") pod \"40a9abc6-a069-44c5-b62a-b700c8994560\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") "
Nov 02 22:33:02.385190 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385150    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-kube-api-access-d56ns\" (UniqueName: \"kubernetes.io/projected/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-kube-api-access-d56ns\") pod \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") "
Nov 02 22:33:02.385190 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385155    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tmp-dir" pod "af6506ab-99cf-4bc1-89b8-3340e6f0c411" (UID: "af6506ab-99cf-4bc1-89b8-3340e6f0c411"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.385190 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385170    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-home" pod "af6506ab-99cf-4bc1-89b8-3340e6f0c411" (UID: "af6506ab-99cf-4bc1-89b8-3340e6f0c411"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.385374 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385190    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-tmp-dir\") pod \"40a9abc6-a069-44c5-b62a-b700c8994560\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") "
Nov 02 22:33:02.385374 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385209    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tls-certs\") pod \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") "
Nov 02 22:33:02.385496 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385376    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-home\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-home\") pod \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") "
Nov 02 22:33:02.385496 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385415    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-home\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-home\") pod \"40a9abc6-a069-44c5-b62a-b700c8994560\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") "
Nov 02 22:33:02.385496 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385413    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-model-cache" pod "40a9abc6-a069-44c5-b62a-b700c8994560" (UID: "40a9abc6-a069-44c5-b62a-b700c8994560"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.385496 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385476    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-model-cache\") pod \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") "
Nov 02 22:33:02.385730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385519    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40a9abc6-a069-44c5-b62a-b700c8994560-tls-certs\" (UniqueName: \"kubernetes.io/secret/40a9abc6-a069-44c5-b62a-b700c8994560-tls-certs\") pod \"40a9abc6-a069-44c5-b62a-b700c8994560\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") "
Nov 02 22:33:02.385730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385534    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-tmp-dir" pod "40a9abc6-a069-44c5-b62a-b700c8994560" (UID: "40a9abc6-a069-44c5-b62a-b700c8994560"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.385730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385559    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-dshm\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-dshm\") pod \"40a9abc6-a069-44c5-b62a-b700c8994560\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") "
Nov 02 22:33:02.385730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385597    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40a9abc6-a069-44c5-b62a-b700c8994560-kube-api-access-cg2mn\" (UniqueName: \"kubernetes.io/projected/40a9abc6-a069-44c5-b62a-b700c8994560-kube-api-access-cg2mn\") pod \"40a9abc6-a069-44c5-b62a-b700c8994560\" (UID: \"40a9abc6-a069-44c5-b62a-b700c8994560\") "
Nov 02 22:33:02.385730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385648    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tmp-dir\") pod \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") "
Nov 02 22:33:02.385730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385677    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-model-cache\") pod \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\" (UID: \"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b\") "
Nov 02 22:33:02.385730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385728    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af6506ab-99cf-4bc1-89b8-3340e6f0c411-kube-api-access-t7q8b\" (UniqueName: \"kubernetes.io/projected/af6506ab-99cf-4bc1-89b8-3340e6f0c411-kube-api-access-t7q8b\") pod \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\" (UID: \"af6506ab-99cf-4bc1-89b8-3340e6f0c411\") "
Nov 02 22:33:02.386678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.386539    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.386678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.386566    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.386678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.386580    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.386678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.386595    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.386678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385727    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-model-cache" pod "af6506ab-99cf-4bc1-89b8-3340e6f0c411" (UID: "af6506ab-99cf-4bc1-89b8-3340e6f0c411"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.386678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385902    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-home" pod "40a9abc6-a069-44c5-b62a-b700c8994560" (UID: "40a9abc6-a069-44c5-b62a-b700c8994560"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.386678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.385946    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-home" pod "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" (UID: "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.386678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.386063    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-model-cache" pod "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" (UID: "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.386678 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.386224    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tmp-dir" pod "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" (UID: "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.387795 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.387768    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-kube-api-access-d56ns" pod "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" (UID: "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b"). InnerVolumeSpecName "kube-api-access-d56ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:02.388246 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.388222    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-dshm" pod "af6506ab-99cf-4bc1-89b8-3340e6f0c411" (UID: "af6506ab-99cf-4bc1-89b8-3340e6f0c411"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.388349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.388274    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-dshm" pod "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" (UID: "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.388349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.388296    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tls-certs" pod "af6506ab-99cf-4bc1-89b8-3340e6f0c411" (UID: "af6506ab-99cf-4bc1-89b8-3340e6f0c411"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:33:02.388349 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.388299    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tls-certs" pod "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" (UID: "f3a2f228-12a8-4e72-b32e-9b7fcb223f0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:33:02.388912 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.388887    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-dshm" pod "40a9abc6-a069-44c5-b62a-b700c8994560" (UID: "40a9abc6-a069-44c5-b62a-b700c8994560"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:02.389088 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.389067    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af6506ab-99cf-4bc1-89b8-3340e6f0c411-kube-api-access-t7q8b" pod "af6506ab-99cf-4bc1-89b8-3340e6f0c411" (UID: "af6506ab-99cf-4bc1-89b8-3340e6f0c411"). InnerVolumeSpecName "kube-api-access-t7q8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:02.389164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.389091    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a9abc6-a069-44c5-b62a-b700c8994560-kube-api-access-cg2mn" pod "40a9abc6-a069-44c5-b62a-b700c8994560" (UID: "40a9abc6-a069-44c5-b62a-b700c8994560"). InnerVolumeSpecName "kube-api-access-cg2mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:02.389522 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.389503    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40a9abc6-a069-44c5-b62a-b700c8994560-tls-certs" pod "40a9abc6-a069-44c5-b62a-b700c8994560" (UID: "40a9abc6-a069-44c5-b62a-b700c8994560"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:33:02.488184 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488152    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d56ns\" (UniqueName: \"kubernetes.io/projected/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-kube-api-access-d56ns\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488184 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488184    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488184 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488194    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488202    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488210    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488219    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40a9abc6-a069-44c5-b62a-b700c8994560-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488226    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40a9abc6-a069-44c5-b62a-b700c8994560-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488234    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg2mn\" (UniqueName: \"kubernetes.io/projected/40a9abc6-a069-44c5-b62a-b700c8994560-kube-api-access-cg2mn\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488242    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488251    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488260    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7q8b\" (UniqueName: \"kubernetes.io/projected/af6506ab-99cf-4bc1-89b8-3340e6f0c411-kube-api-access-t7q8b\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488268    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488276    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af6506ab-99cf-4bc1-89b8-3340e6f0c411-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.488442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.488283    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af6506ab-99cf-4bc1-89b8-3340e6f0c411-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:02.792071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.792037    5067 generic.go:356] "Generic (PLEG): container finished" podID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerID="028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695" exitCode=0
Nov 02 22:33:02.792569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.792083    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"
Nov 02 22:33:02.792569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.792114    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" event={"ID":"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b","Type":"ContainerDied","Data":"028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695"}
Nov 02 22:33:02.792569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.792157    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js" event={"ID":"f3a2f228-12a8-4e72-b32e-9b7fcb223f0b","Type":"ContainerDied","Data":"2a53769cb5b12fc4fc89cade12e6198e1d581b10fca809bd014455286f759873"}
Nov 02 22:33:02.792569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.792180    5067 scope.go:122] "RemoveContainer" containerID="028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695"
Nov 02 22:33:02.793686 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.793665    5067 generic.go:356] "Generic (PLEG): container finished" podID="40a9abc6-a069-44c5-b62a-b700c8994560" containerID="aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39" exitCode=0
Nov 02 22:33:02.793784 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.793705    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"
Nov 02 22:33:02.793784 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.793744    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" event={"ID":"40a9abc6-a069-44c5-b62a-b700c8994560","Type":"ContainerDied","Data":"aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39"}
Nov 02 22:33:02.793884 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.793792    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94" event={"ID":"40a9abc6-a069-44c5-b62a-b700c8994560","Type":"ContainerDied","Data":"dd7be79d27ee5fc22934eaa1917dced336d13b85eb354c644e3897ebdde53998"}
Nov 02 22:33:02.795178 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.795157    5067 generic.go:356] "Generic (PLEG): container finished" podID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerID="636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23" exitCode=0
Nov 02 22:33:02.795285 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.795224    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"
Nov 02 22:33:02.795285 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.795245    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" event={"ID":"af6506ab-99cf-4bc1-89b8-3340e6f0c411","Type":"ContainerDied","Data":"636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23"}
Nov 02 22:33:02.795428 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.795297    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg" event={"ID":"af6506ab-99cf-4bc1-89b8-3340e6f0c411","Type":"ContainerDied","Data":"abfa18a8b6f05913340087f9bf81ecfe3d11fb46501fa8ed6b9021376956f3bc"}
Nov 02 22:33:02.804570 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.804306    5067 scope.go:122] "RemoveContainer" containerID="028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695"
Nov 02 22:33:02.804702 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:02.804676    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695\": container with ID starting with 028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695 not found: ID does not exist" containerID="028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695"
Nov 02 22:33:02.804759 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.804716    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695"} err="failed to get container status \"028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695\": rpc error: code = NotFound desc = could not find container \"028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695\": container with ID starting with 028efa3bb5b59eb04d85e39825cf2d2a609334853c22399015db0bf586e38695 not found: ID does not exist"
Nov 02 22:33:02.804759 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.804740    5067 scope.go:122] "RemoveContainer" containerID="aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39"
Nov 02 22:33:02.819647 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.819614    5067 scope.go:122] "RemoveContainer" containerID="aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39"
Nov 02 22:33:02.820035 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:02.820004    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39\": container with ID starting with aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39 not found: ID does not exist" containerID="aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39"
Nov 02 22:33:02.820136 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.820045    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39"} err="failed to get container status \"aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39\": rpc error: code = NotFound desc = could not find container \"aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39\": container with ID starting with aa8de160c0d676149ff4e8b2d5bbd99a6b87d5adf5364d34e6b7d197a6426c39 not found: ID does not exist"
Nov 02 22:33:02.820136 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.820089    5067 scope.go:122] "RemoveContainer" containerID="636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23"
Nov 02 22:33:02.832157 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.832131    5067 scope.go:122] "RemoveContainer" containerID="636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23"
Nov 02 22:33:02.832612 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:02.832536    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23\": container with ID starting with 636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23 not found: ID does not exist" containerID="636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23"
Nov 02 22:33:02.832612 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.832581    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23"} err="failed to get container status \"636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23\": rpc error: code = NotFound desc = could not find container \"636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23\": container with ID starting with 636d2395858e1ccdedac7d19180819700a0b1a20ca525c97b9e14d4d11693c23 not found: ID does not exist"
Nov 02 22:33:02.971445 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.971417    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"]
Nov 02 22:33:02.979866 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:02.979837    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/premium-w5-fe810f-kserve-84cf95ff65-4jb94"]
Nov 02 22:33:03.017020 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.016989    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:33:03.019675 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.019655    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:33:03.082636 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.082569    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"]
Nov 02 22:33:03.085574 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.085552    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/distinct-w5-fe810f-kserve-85c86cb444-7z8js"]
Nov 02 22:33:03.096193 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.096164    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-xks-ca\") pod \"5feca907-056a-416b-8cf7-246fc20b2194\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") "
Nov 02 22:33:03.096306 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.096292    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5feca907-056a-416b-8cf7-246fc20b2194-kube-api-access-gpvt2\" (UniqueName: \"kubernetes.io/projected/5feca907-056a-416b-8cf7-246fc20b2194-kube-api-access-gpvt2\") pod \"5feca907-056a-416b-8cf7-246fc20b2194\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") "
Nov 02 22:33:03.096400 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.096331    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-xks-ca\") pod \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") "
Nov 02 22:33:03.096400 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.096351    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-openshift-ca\") pod \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") "
Nov 02 22:33:03.096477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.096423    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-kube-api-access-pnm9q\" (UniqueName: \"kubernetes.io/projected/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-kube-api-access-pnm9q\") pod \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\" (UID: \"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0\") "
Nov 02 22:33:03.096477 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.096447    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-openshift-ca\") pod \"5feca907-056a-416b-8cf7-246fc20b2194\" (UID: \"5feca907-056a-416b-8cf7-246fc20b2194\") "
Nov 02 22:33:03.096832 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.096804    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-openshift-ca" pod "9b0584fc-060b-4c59-a5ff-34a7ae51b1d0" (UID: "9b0584fc-060b-4c59-a5ff-34a7ae51b1d0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:03.096950 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.096876    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-openshift-ca" pod "5feca907-056a-416b-8cf7-246fc20b2194" (UID: "5feca907-056a-416b-8cf7-246fc20b2194"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:03.097061 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.097040    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-openshift-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:03.097061 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.097063    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-openshift-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:03.098575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.098552    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5feca907-056a-416b-8cf7-246fc20b2194-kube-api-access-gpvt2" pod "5feca907-056a-416b-8cf7-246fc20b2194" (UID: "5feca907-056a-416b-8cf7-246fc20b2194"). InnerVolumeSpecName "kube-api-access-gpvt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:03.098654 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.098571    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-kube-api-access-pnm9q" pod "9b0584fc-060b-4c59-a5ff-34a7ae51b1d0" (UID: "9b0584fc-060b-4c59-a5ff-34a7ae51b1d0"). InnerVolumeSpecName "kube-api-access-pnm9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:03.098812 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.098796    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-xks-ca" pod "5feca907-056a-416b-8cf7-246fc20b2194" (UID: "5feca907-056a-416b-8cf7-246fc20b2194"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:33:03.098855 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.098818    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-xks-ca" pod "9b0584fc-060b-4c59-a5ff-34a7ae51b1d0" (UID: "9b0584fc-060b-4c59-a5ff-34a7ae51b1d0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:33:03.187621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.187446    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"]
Nov 02 22:33:03.191659 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.191624    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-fe810f/simulator-w5-fe810f-kserve-5dc6fd787b-9b6zg"]
Nov 02 22:33:03.198463 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.198434    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpvt2\" (UniqueName: \"kubernetes.io/projected/5feca907-056a-416b-8cf7-246fc20b2194-kube-api-access-gpvt2\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:03.198575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.198467    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-maas-api-xks-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:03.198575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.198484    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnm9q\" (UniqueName: \"kubernetes.io/projected/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0-kube-api-access-pnm9q\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:03.198575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.198499    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5feca907-056a-416b-8cf7-246fc20b2194-maas-api-xks-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:03.806411 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.806355    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"
Nov 02 22:33:03.806411 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.806390    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb" event={"ID":"9b0584fc-060b-4c59-a5ff-34a7ae51b1d0","Type":"ContainerDied","Data":"305c198e6ec80dfa74f7bacda8f483cb8114ef2f2323aace769e2b126aad9f21"}
Nov 02 22:33:03.806982 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.806473    5067 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="305c198e6ec80dfa74f7bacda8f483cb8114ef2f2323aace769e2b126aad9f21"
Nov 02 22:33:03.809094 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.809069    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"
Nov 02 22:33:03.809215 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.809094    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn" event={"ID":"5feca907-056a-416b-8cf7-246fc20b2194","Type":"ContainerDied","Data":"0e86d19e63dd39c3e91b3edde89e0db3b837ecab386d67a297478e3777b821a9"}
Nov 02 22:33:03.809215 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.809138    5067 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e86d19e63dd39c3e91b3edde89e0db3b837ecab386d67a297478e3777b821a9"
Nov 02 22:33:03.848516 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.848481    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" path="/var/lib/kubelet/pods/40a9abc6-a069-44c5-b62a-b700c8994560/volumes"
Nov 02 22:33:03.849092 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.849071    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" path="/var/lib/kubelet/pods/af6506ab-99cf-4bc1-89b8-3340e6f0c411/volumes"
Nov 02 22:33:03.849629 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:03.849605    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" path="/var/lib/kubelet/pods/f3a2f228-12a8-4e72-b32e-9b7fcb223f0b/volumes"
Nov 02 22:33:07.581354 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:07.581276    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:33:09.135069 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:09.128584    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"]
Nov 02 22:33:09.135069 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:09.128774    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy" containerID="cri-o://35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f" gracePeriod=30
Nov 02 22:33:09.160448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:09.160414    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"]
Nov 02 22:33:09.160694 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:09.160667    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerName="istio-proxy" containerID="cri-o://921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056" gracePeriod=30
Nov 02 22:33:09.166331 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:09.166299    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"]
Nov 02 22:33:09.166553 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:09.166527    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy" containerID="cri-o://2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b" gracePeriod=30
Nov 02 22:33:14.477132 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.476712    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:33:14.486271 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.485916    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-kube-api-access-mcn4m\" (UniqueName: \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-kube-api-access-mcn4m\") pod \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") "
Nov 02 22:33:14.486271 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.486163    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-certs\") pod \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") "
Nov 02 22:33:14.486271 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.486212    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-data\") pod \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") "
Nov 02 22:33:14.486558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.486300    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-credential-socket\") pod \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") "
Nov 02 22:33:14.486558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.486349    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-podinfo\") pod \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") "
Nov 02 22:33:14.486558 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.486549    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-envoy\") pod \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") "
Nov 02 22:33:14.486709 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.486591    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-socket\") pod \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") "
Nov 02 22:33:14.486709 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.486678    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istiod-ca-cert\") pod \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") "
Nov 02 22:33:14.486807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.486795    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-token\" (UniqueName: \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-token\") pod \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\" (UID: \"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a\") "
Nov 02 22:33:14.487512 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.487435    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-data" pod "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" (UID: "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.487766 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.487627    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-certs" pod "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" (UID: "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.487856 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.487792    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-credential-socket" pod "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" (UID: "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.487926 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.487858    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-socket" pod "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" (UID: "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.488094 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.488061    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istiod-ca-cert" pod "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" (UID: "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:14.490583 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.490019    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-podinfo" pod "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" (UID: "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:14.495412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.495057    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-envoy" pod "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" (UID: "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.495412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.495231    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-kube-api-access-mcn4m" pod "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" (UID: "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a"). InnerVolumeSpecName "kube-api-access-mcn4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:14.495412 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.495255    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-token" pod "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" (UID: "0068f3ef-ef4d-47d1-a79b-9fa8ea69461a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:14.509335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.509311    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:33:14.549559 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.549530    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:33:14.588629 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588597    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-socket\") pod \"e68eb745-1a75-4eaf-9569-54624bd97223\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") "
Nov 02 22:33:14.588807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588651    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-kube-api-access-b4z87\" (UniqueName: \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-kube-api-access-b4z87\") pod \"e68eb745-1a75-4eaf-9569-54624bd97223\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") "
Nov 02 22:33:14.588807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588677    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e68eb745-1a75-4eaf-9569-54624bd97223-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e68eb745-1a75-4eaf-9569-54624bd97223-istio-podinfo\") pod \"e68eb745-1a75-4eaf-9569-54624bd97223\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") "
Nov 02 22:33:14.588807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588717    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-data\") pod \"0b203dd1-35c7-42d9-993d-d515b473a2f1\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") "
Nov 02 22:33:14.588807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588740    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-credential-socket\") pod \"e68eb745-1a75-4eaf-9569-54624bd97223\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") "
Nov 02 22:33:14.588807 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588767    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-istio-token\" (UniqueName: \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-istio-token\") pod \"e68eb745-1a75-4eaf-9569-54624bd97223\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") "
Nov 02 22:33:14.589100 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588825    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-token\" (UniqueName: \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-token\") pod \"0b203dd1-35c7-42d9-993d-d515b473a2f1\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") "
Nov 02 22:33:14.589100 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588873    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-data\") pod \"e68eb745-1a75-4eaf-9569-54624bd97223\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") "
Nov 02 22:33:14.589100 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588942    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-podinfo\") pod \"0b203dd1-35c7-42d9-993d-d515b473a2f1\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") "
Nov 02 22:33:14.589100 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588988    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e68eb745-1a75-4eaf-9569-54624bd97223-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e68eb745-1a75-4eaf-9569-54624bd97223-istiod-ca-cert\") pod \"e68eb745-1a75-4eaf-9569-54624bd97223\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") "
Nov 02 22:33:14.589100 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.588990    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-socket" pod "e68eb745-1a75-4eaf-9569-54624bd97223" (UID: "e68eb745-1a75-4eaf-9569-54624bd97223"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.589100 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589019    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-socket\") pod \"0b203dd1-35c7-42d9-993d-d515b473a2f1\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") "
Nov 02 22:33:14.589100 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589042    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-data" pod "0b203dd1-35c7-42d9-993d-d515b473a2f1" (UID: "0b203dd1-35c7-42d9-993d-d515b473a2f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.589100 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589064    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-envoy\") pod \"e68eb745-1a75-4eaf-9569-54624bd97223\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") "
Nov 02 22:33:14.589539 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589110    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-credential-socket\") pod \"0b203dd1-35c7-42d9-993d-d515b473a2f1\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") "
Nov 02 22:33:14.589539 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589165    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b203dd1-35c7-42d9-993d-d515b473a2f1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b203dd1-35c7-42d9-993d-d515b473a2f1-istiod-ca-cert\") pod \"0b203dd1-35c7-42d9-993d-d515b473a2f1\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") "
Nov 02 22:33:14.589539 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589194    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-data" pod "e68eb745-1a75-4eaf-9569-54624bd97223" (UID: "e68eb745-1a75-4eaf-9569-54624bd97223"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.589539 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589221    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-certs\") pod \"0b203dd1-35c7-42d9-993d-d515b473a2f1\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") "
Nov 02 22:33:14.589539 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589272    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-kube-api-access-nvdv8\" (UniqueName: \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-kube-api-access-nvdv8\") pod \"0b203dd1-35c7-42d9-993d-d515b473a2f1\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") "
Nov 02 22:33:14.589539 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589302    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-certs\") pod \"e68eb745-1a75-4eaf-9569-54624bd97223\" (UID: \"e68eb745-1a75-4eaf-9569-54624bd97223\") "
Nov 02 22:33:14.589539 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589337    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-envoy\") pod \"0b203dd1-35c7-42d9-993d-d515b473a2f1\" (UID: \"0b203dd1-35c7-42d9-993d-d515b473a2f1\") "
Nov 02 22:33:14.589905 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589652    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-socket" pod "0b203dd1-35c7-42d9-993d-d515b473a2f1" (UID: "0b203dd1-35c7-42d9-993d-d515b473a2f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.589905 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589694    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e68eb745-1a75-4eaf-9569-54624bd97223-istiod-ca-cert" pod "e68eb745-1a75-4eaf-9569-54624bd97223" (UID: "e68eb745-1a75-4eaf-9569-54624bd97223"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:14.590022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.589962    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-credential-socket" pod "e68eb745-1a75-4eaf-9569-54624bd97223" (UID: "e68eb745-1a75-4eaf-9569-54624bd97223"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.590105 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590086    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e68eb745-1a75-4eaf-9569-54624bd97223-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590247 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590230    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590253    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590269    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcn4m\" (UniqueName: \"kubernetes.io/projected/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-kube-api-access-mcn4m\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590285    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590300    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590315    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590329    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590343    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590358    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590403    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590417    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590431    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590446    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590459    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590091    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b203dd1-35c7-42d9-993d-d515b473a2f1-istiod-ca-cert" pod "0b203dd1-35c7-42d9-993d-d515b473a2f1" (UID: "0b203dd1-35c7-42d9-993d-d515b473a2f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590269    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-certs" pod "0b203dd1-35c7-42d9-993d-d515b473a2f1" (UID: "0b203dd1-35c7-42d9-993d-d515b473a2f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590414    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-certs" pod "e68eb745-1a75-4eaf-9569-54624bd97223" (UID: "e68eb745-1a75-4eaf-9569-54624bd97223"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.590589 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.590496    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-credential-socket" pod "0b203dd1-35c7-42d9-993d-d515b473a2f1" (UID: "0b203dd1-35c7-42d9-993d-d515b473a2f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.591922 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.591897    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-podinfo" pod "0b203dd1-35c7-42d9-993d-d515b473a2f1" (UID: "0b203dd1-35c7-42d9-993d-d515b473a2f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:14.592207 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.592163    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e68eb745-1a75-4eaf-9569-54624bd97223-istio-podinfo" pod "e68eb745-1a75-4eaf-9569-54624bd97223" (UID: "e68eb745-1a75-4eaf-9569-54624bd97223"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:14.592329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.592216    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-token" pod "0b203dd1-35c7-42d9-993d-d515b473a2f1" (UID: "0b203dd1-35c7-42d9-993d-d515b473a2f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:14.592427 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.592339    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-envoy" pod "0b203dd1-35c7-42d9-993d-d515b473a2f1" (UID: "0b203dd1-35c7-42d9-993d-d515b473a2f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.592627 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.592596    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-envoy" pod "e68eb745-1a75-4eaf-9569-54624bd97223" (UID: "e68eb745-1a75-4eaf-9569-54624bd97223"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:14.592727 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.592600    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-kube-api-access-b4z87" pod "e68eb745-1a75-4eaf-9569-54624bd97223" (UID: "e68eb745-1a75-4eaf-9569-54624bd97223"). InnerVolumeSpecName "kube-api-access-b4z87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:14.593481 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.593458    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-istio-token" pod "e68eb745-1a75-4eaf-9569-54624bd97223" (UID: "e68eb745-1a75-4eaf-9569-54624bd97223"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:14.593608 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.593585    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-kube-api-access-nvdv8" pod "0b203dd1-35c7-42d9-993d-d515b473a2f1" (UID: "0b203dd1-35c7-42d9-993d-d515b473a2f1"). InnerVolumeSpecName "kube-api-access-nvdv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:14.692538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692454    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4z87\" (UniqueName: \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-kube-api-access-b4z87\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692494    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e68eb745-1a75-4eaf-9569-54624bd97223-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692509    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e68eb745-1a75-4eaf-9569-54624bd97223-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692523    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692537    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692827 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692550    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692827 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692564    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692827 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692578    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b203dd1-35c7-42d9-993d-d515b473a2f1-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692827 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692592    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692827 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692605    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvdv8\" (UniqueName: \"kubernetes.io/projected/0b203dd1-35c7-42d9-993d-d515b473a2f1-kube-api-access-nvdv8\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692827 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692619    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e68eb745-1a75-4eaf-9569-54624bd97223-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.692827 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.692636    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b203dd1-35c7-42d9-993d-d515b473a2f1-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:14.900782 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.900741    5067 generic.go:356] "Generic (PLEG): container finished" podID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerID="921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056" exitCode=0
Nov 02 22:33:14.901069 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.900783    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" event={"ID":"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a","Type":"ContainerDied","Data":"921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056"}
Nov 02 22:33:14.901240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.901112    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj" event={"ID":"0068f3ef-ef4d-47d1-a79b-9fa8ea69461a","Type":"ContainerDied","Data":"847363aaf83d9c598007c1645a7b77b68491be1666feb8040974460c687aa24f"}
Nov 02 22:33:14.901240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.900792    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"
Nov 02 22:33:14.901240 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.901150    5067 scope.go:122] "RemoveContainer" containerID="921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056"
Nov 02 22:33:14.903443 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.903359    5067 generic.go:356] "Generic (PLEG): container finished" podID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerID="35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f" exitCode=0
Nov 02 22:33:14.903605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.903547    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"
Nov 02 22:33:14.903605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.903568    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" event={"ID":"0b203dd1-35c7-42d9-993d-d515b473a2f1","Type":"ContainerDied","Data":"35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f"}
Nov 02 22:33:14.903713 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.903619    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q" event={"ID":"0b203dd1-35c7-42d9-993d-d515b473a2f1","Type":"ContainerDied","Data":"ab35b412dd2e78f46d2a593f4a06d30c2db32b85b855ed33684cc3e0dda75475"}
Nov 02 22:33:14.905963 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.905400    5067 generic.go:356] "Generic (PLEG): container finished" podID="e68eb745-1a75-4eaf-9569-54624bd97223" containerID="2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b" exitCode=0
Nov 02 22:33:14.905963 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.905450    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" event={"ID":"e68eb745-1a75-4eaf-9569-54624bd97223","Type":"ContainerDied","Data":"2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b"}
Nov 02 22:33:14.905963 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.905491    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs" event={"ID":"e68eb745-1a75-4eaf-9569-54624bd97223","Type":"ContainerDied","Data":"6f7301d182f1518a67f87f651fdd96b409ca2eb6b924b065fe23dada7d2ba114"}
Nov 02 22:33:14.905963 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.905562    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"
Nov 02 22:33:14.922067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.922047    5067 scope.go:122] "RemoveContainer" containerID="921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056"
Nov 02 22:33:14.922432 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:14.922413    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056\": container with ID starting with 921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056 not found: ID does not exist" containerID="921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056"
Nov 02 22:33:14.922486 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.922442    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056"} err="failed to get container status \"921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056\": rpc error: code = NotFound desc = could not find container \"921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056\": container with ID starting with 921e6a363b8e2698687e881fe4414d42b63693f91e19bff91c72956fd1131056 not found: ID does not exist"
Nov 02 22:33:14.922486 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.922461    5067 scope.go:122] "RemoveContainer" containerID="35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f"
Nov 02 22:33:14.933850 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.933826    5067 scope.go:122] "RemoveContainer" containerID="35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f"
Nov 02 22:33:14.934184 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:14.934164    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f\": container with ID starting with 35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f not found: ID does not exist" containerID="35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f"
Nov 02 22:33:14.934250 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.934193    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f"} err="failed to get container status \"35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f\": rpc error: code = NotFound desc = could not find container \"35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f\": container with ID starting with 35cf14998a7bc340e9c4d4e9c11e9aae7ab603775537daca8d23536cfb05c77f not found: ID does not exist"
Nov 02 22:33:14.934250 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.934216    5067 scope.go:122] "RemoveContainer" containerID="2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b"
Nov 02 22:33:14.943495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.943474    5067 scope.go:122] "RemoveContainer" containerID="2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b"
Nov 02 22:33:14.943776 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:14.943754    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b\": container with ID starting with 2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b not found: ID does not exist" containerID="2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b"
Nov 02 22:33:14.943855 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:14.943787    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b"} err="failed to get container status \"2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b\": rpc error: code = NotFound desc = could not find container \"2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b\": container with ID starting with 2370d6c5e045213294f2ffa273dae673acdf6a60bb5ba64e9afca13455cd5f7b not found: ID does not exist"
Nov 02 22:33:15.057805 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:15.057767    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"]
Nov 02 22:33:15.060837 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:15.060810    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-jhphs"]
Nov 02 22:33:15.166732 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:15.166606    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"]
Nov 02 22:33:15.170214 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:15.170184    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-x4f5q"]
Nov 02 22:33:15.271619 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:15.271576    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"]
Nov 02 22:33:15.275804 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:15.275776    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ca85db-openshift-default-6bdff5f4df-76zzj"]
Nov 02 22:33:15.846463 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:15.846420    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" path="/var/lib/kubelet/pods/0068f3ef-ef4d-47d1-a79b-9fa8ea69461a/volumes"
Nov 02 22:33:15.847132 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:15.847109    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" path="/var/lib/kubelet/pods/0b203dd1-35c7-42d9-993d-d515b473a2f1/volumes"
Nov 02 22:33:15.847651 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:15.847630    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" path="/var/lib/kubelet/pods/e68eb745-1a75-4eaf-9569-54624bd97223/volumes"
Nov 02 22:33:16.431731 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.431695    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"]
Nov 02 22:33:16.431960 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.431927    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp" podUID="f3880123-ed80-4cec-8369-998133747622" containerName="maas-api" containerID="cri-o://badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7" gracePeriod=30
Nov 02 22:33:16.805606 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.805572    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:33:16.819637 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.819589    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3880123-ed80-4cec-8369-998133747622-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3880123-ed80-4cec-8369-998133747622-openshift-service-ca\") pod \"f3880123-ed80-4cec-8369-998133747622\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") "
Nov 02 22:33:16.819850 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.819657    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-maas-api-tls\") pod \"f3880123-ed80-4cec-8369-998133747622\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") "
Nov 02 22:33:16.819850 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.819701    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-metrics-certs\") pod \"f3880123-ed80-4cec-8369-998133747622\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") "
Nov 02 22:33:16.820005 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.819944    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3880123-ed80-4cec-8369-998133747622-kube-api-access-8rqqj\" (UniqueName: \"kubernetes.io/projected/f3880123-ed80-4cec-8369-998133747622-kube-api-access-8rqqj\") pod \"f3880123-ed80-4cec-8369-998133747622\" (UID: \"f3880123-ed80-4cec-8369-998133747622\") "
Nov 02 22:33:16.820245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.820219    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3880123-ed80-4cec-8369-998133747622-openshift-service-ca" pod "f3880123-ed80-4cec-8369-998133747622" (UID: "f3880123-ed80-4cec-8369-998133747622"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:16.824677 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.820618    5067 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3880123-ed80-4cec-8369-998133747622-openshift-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:16.839055 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.838951    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-maas-api-tls" pod "f3880123-ed80-4cec-8369-998133747622" (UID: "f3880123-ed80-4cec-8369-998133747622"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:33:16.870390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.848572    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3880123-ed80-4cec-8369-998133747622-kube-api-access-8rqqj" pod "f3880123-ed80-4cec-8369-998133747622" (UID: "f3880123-ed80-4cec-8369-998133747622"). InnerVolumeSpecName "kube-api-access-8rqqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:16.870390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.857582    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-metrics-certs" pod "f3880123-ed80-4cec-8369-998133747622" (UID: "f3880123-ed80-4cec-8369-998133747622"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:33:16.922387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.922143    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rqqj\" (UniqueName: \"kubernetes.io/projected/f3880123-ed80-4cec-8369-998133747622-kube-api-access-8rqqj\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:16.922387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.922180    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-maas-api-tls\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:16.922387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.922195    5067 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3880123-ed80-4cec-8369-998133747622-metrics-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:16.930194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.930143    5067 generic.go:356] "Generic (PLEG): container finished" podID="f3880123-ed80-4cec-8369-998133747622" containerID="badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7" exitCode=0
Nov 02 22:33:16.930654 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.930615    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp" event={"ID":"f3880123-ed80-4cec-8369-998133747622","Type":"ContainerDied","Data":"badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7"}
Nov 02 22:33:16.931291 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.931259    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp" event={"ID":"f3880123-ed80-4cec-8369-998133747622","Type":"ContainerDied","Data":"6a5f36c970ac1c2b66e0d8ac06669cff16113fc90f55a93220a1abfd03be0816"}
Nov 02 22:33:16.931514 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.931497    5067 scope.go:122] "RemoveContainer" containerID="badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7"
Nov 02 22:33:16.931852 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.931767    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"
Nov 02 22:33:16.953814 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.953769    5067 scope.go:122] "RemoveContainer" containerID="badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7"
Nov 02 22:33:16.954266 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:16.954232    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7\": container with ID starting with badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7 not found: ID does not exist" containerID="badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7"
Nov 02 22:33:16.954353 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:16.954281    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7"} err="failed to get container status \"badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7\": rpc error: code = NotFound desc = could not find container \"badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7\": container with ID starting with badda02a5fb8f8a368073f14a2be71e4319fd816e8279a1798a28ce9261495e7 not found: ID does not exist"
Nov 02 22:33:17.117057 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:17.116934    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"]
Nov 02 22:33:17.125968 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:17.125928    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-49a557-849f49fb54-g95dp"]
Nov 02 22:33:17.908708 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:17.908658    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3880123-ed80-4cec-8369-998133747622" path="/var/lib/kubelet/pods/f3880123-ed80-4cec-8369-998133747622/volumes"
Nov 02 22:33:25.723016 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:25.722971    5067 pod_container_manager_linux.go:218] "Failed to delete cgroup paths" cgroupName=["kubepods","burstable","pode5ac378c-d84e-4bc1-9fd6-595af7ad3f97"] err="unable to destroy cgroup paths for cgroup [kubepods burstable pode5ac378c-d84e-4bc1-9fd6-595af7ad3f97] : Timed out while waiting for systemd to remove kubepods-burstable-pode5ac378c_d84e_4bc1_9fd6_595af7ad3f97.slice"
Nov 02 22:33:25.723016 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:25.723015    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods burstable pode5ac378c-d84e-4bc1-9fd6-595af7ad3f97] : unable to destroy cgroup paths for cgroup [kubepods burstable pode5ac378c-d84e-4bc1-9fd6-595af7ad3f97] : Timed out while waiting for systemd to remove kubepods-burstable-pode5ac378c_d84e_4bc1_9fd6_595af7ad3f97.slice" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97"
Nov 02 22:33:26.021741 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:26.021659    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"
Nov 02 22:33:26.247290 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:26.247235    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"]
Nov 02 22:33:26.283622 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:26.283524    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3224ea-openshift-default-7c9cb65579-wcxn2"]
Nov 02 22:33:27.033101 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.033069    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"]
Nov 02 22:33:27.033713 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.033681    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy" containerID="cri-o://d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa" gracePeriod=30
Nov 02 22:33:27.057303 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.057269    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"]
Nov 02 22:33:27.057534 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.057508    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy" containerID="cri-o://94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06" gracePeriod=30
Nov 02 22:33:27.068901 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.068861    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"]
Nov 02 22:33:27.069235 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.069163    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy" containerID="cri-o://11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2" gracePeriod=30
Nov 02 22:33:27.696000 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.695966    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"]
Nov 02 22:33:27.696536 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.696491    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy" containerID="cri-o://f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397" gracePeriod=30
Nov 02 22:33:27.702194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.700249    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"]
Nov 02 22:33:27.702194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.700492    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy" containerID="cri-o://b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5" gracePeriod=30
Nov 02 22:33:27.707906 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.707886    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"]
Nov 02 22:33:27.708211 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.708185    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy" containerID="cri-o://e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b" gracePeriod=30
Nov 02 22:33:27.849964 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:27.849923    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5ac378c-d84e-4bc1-9fd6-595af7ad3f97" path="/var/lib/kubelet/pods/e5ac378c-d84e-4bc1-9fd6-595af7ad3f97/volumes"
Nov 02 22:33:29.713414 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:29.709737    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"]
Nov 02 22:33:29.713414 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:29.709993    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy" containerID="cri-o://14da73a16794fd9cb0a6e1f33bd7e15e69d03d55a040b10e2af573d382662852" gracePeriod=30
Nov 02 22:33:29.714019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:29.713849    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"]
Nov 02 22:33:29.714107 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:29.714060    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy" containerID="cri-o://29a6bf44b255c7af1371bddd91f93f3dbae162f323070dc2e0c53c20a6c7f547" gracePeriod=30
Nov 02 22:33:29.742540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:29.742501    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"]
Nov 02 22:33:29.742789 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:29.742760    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" containerID="cri-o://3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52" gracePeriod=30
Nov 02 22:33:31.748973 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:31.748938    5067 patch_prober.go:29] interesting pod/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:33:31.749475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:31.748990    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:33:32.360954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.360567    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:33:32.389359 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.389335    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:33:32.419687 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.419663    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:33:32.430770 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.430748    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-kube-api-access-g64rv\" (UniqueName: \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-kube-api-access-g64rv\") pod \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") "
Nov 02 22:33:32.430873 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.430780    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-token\" (UniqueName: \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-token\") pod \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") "
Nov 02 22:33:32.430873 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.430819    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-data\") pod \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") "
Nov 02 22:33:32.431014 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.430988    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istiod-ca-cert\") pod \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") "
Nov 02 22:33:32.431065 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431051    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-credential-socket\") pod \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") "
Nov 02 22:33:32.431065 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431052    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-data" pod "e71caf86-9d04-4442-bcfd-5b7bf21900e6" (UID: "e71caf86-9d04-4442-bcfd-5b7bf21900e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.431173 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431077    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-kube-api-access-lgzqz\" (UniqueName: \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-kube-api-access-lgzqz\") pod \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") "
Nov 02 22:33:32.431173 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431106    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-token\" (UniqueName: \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-token\") pod \"59ed629b-1c29-4cfc-909c-7b4089c141ac\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") "
Nov 02 22:33:32.431173 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431125    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-data\") pod \"59ed629b-1c29-4cfc-909c-7b4089c141ac\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") "
Nov 02 22:33:32.431173 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431150    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-certs\") pod \"59ed629b-1c29-4cfc-909c-7b4089c141ac\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") "
Nov 02 22:33:32.431402 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431213    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-data\") pod \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") "
Nov 02 22:33:32.431402 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431251    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-socket\") pod \"59ed629b-1c29-4cfc-909c-7b4089c141ac\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") "
Nov 02 22:33:32.431402 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431290    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-podinfo\") pod \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") "
Nov 02 22:33:32.431402 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431323    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-credential-socket" pod "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" (UID: "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.431402 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431378    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-envoy\") pod \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") "
Nov 02 22:33:32.431668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431418    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-podinfo\") pod \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") "
Nov 02 22:33:32.431668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431472    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-socket\") pod \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") "
Nov 02 22:33:32.431668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431542    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-certs\") pod \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") "
Nov 02 22:33:32.431668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431592    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-kube-api-access-4ffnv\" (UniqueName: \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-kube-api-access-4ffnv\") pod \"59ed629b-1c29-4cfc-909c-7b4089c141ac\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") "
Nov 02 22:33:32.431668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431415    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istiod-ca-cert" pod "e71caf86-9d04-4442-bcfd-5b7bf21900e6" (UID: "e71caf86-9d04-4442-bcfd-5b7bf21900e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:32.431668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431562    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-data" pod "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" (UID: "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.431668 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431580    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-socket" pod "59ed629b-1c29-4cfc-909c-7b4089c141ac" (UID: "59ed629b-1c29-4cfc-909c-7b4089c141ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.432050 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.431818    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-data" pod "59ed629b-1c29-4cfc-909c-7b4089c141ac" (UID: "59ed629b-1c29-4cfc-909c-7b4089c141ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.432111 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432047    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-certs" pod "59ed629b-1c29-4cfc-909c-7b4089c141ac" (UID: "59ed629b-1c29-4cfc-909c-7b4089c141ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432220    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-certs\") pod \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") "
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432256    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-socket" pod "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" (UID: "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432270    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-credential-socket\") pod \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") "
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432324    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-envoy\") pod \"59ed629b-1c29-4cfc-909c-7b4089c141ac\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") "
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432413    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59ed629b-1c29-4cfc-909c-7b4089c141ac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ed629b-1c29-4cfc-909c-7b4089c141ac-istiod-ca-cert\") pod \"59ed629b-1c29-4cfc-909c-7b4089c141ac\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") "
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432447    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-podinfo\") pod \"59ed629b-1c29-4cfc-909c-7b4089c141ac\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") "
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432482    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-token\" (UniqueName: \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-token\") pod \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") "
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432516    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istiod-ca-cert\") pod \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\" (UID: \"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6\") "
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432580    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-credential-socket\") pod \"59ed629b-1c29-4cfc-909c-7b4089c141ac\" (UID: \"59ed629b-1c29-4cfc-909c-7b4089c141ac\") "
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432415    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-certs" pod "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" (UID: "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432702    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-envoy\") pod \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") "
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432747    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-socket\") pod \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\" (UID: \"e71caf86-9d04-4442-bcfd-5b7bf21900e6\") "
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432801    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-credential-socket" pod "59ed629b-1c29-4cfc-909c-7b4089c141ac" (UID: "59ed629b-1c29-4cfc-909c-7b4089c141ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.433067 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433025    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59ed629b-1c29-4cfc-909c-7b4089c141ac-istiod-ca-cert" pod "59ed629b-1c29-4cfc-909c-7b4089c141ac" (UID: "59ed629b-1c29-4cfc-909c-7b4089c141ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433292    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-certs" pod "e71caf86-9d04-4442-bcfd-5b7bf21900e6" (UID: "e71caf86-9d04-4442-bcfd-5b7bf21900e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.432547    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-credential-socket" pod "e71caf86-9d04-4442-bcfd-5b7bf21900e6" (UID: "e71caf86-9d04-4442-bcfd-5b7bf21900e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433488    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istiod-ca-cert" pod "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" (UID: "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433498    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433532    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433549    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433563    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433580    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433598    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433615    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433630    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433643    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433656    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433670    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433683    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ed629b-1c29-4cfc-909c-7b4089c141ac-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433696    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.434024 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.433957    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-kube-api-access-g64rv" pod "e71caf86-9d04-4442-bcfd-5b7bf21900e6" (UID: "e71caf86-9d04-4442-bcfd-5b7bf21900e6"). InnerVolumeSpecName "kube-api-access-g64rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:32.434967 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.434129    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-socket" pod "e71caf86-9d04-4442-bcfd-5b7bf21900e6" (UID: "e71caf86-9d04-4442-bcfd-5b7bf21900e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.435538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.435439    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-envoy" pod "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" (UID: "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.435538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.435507    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-token" pod "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" (UID: "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:32.435749 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.435728    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-podinfo" pod "e71caf86-9d04-4442-bcfd-5b7bf21900e6" (UID: "e71caf86-9d04-4442-bcfd-5b7bf21900e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:32.435914 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.435894    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-podinfo" pod "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" (UID: "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:32.436002 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.435961    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-kube-api-access-lgzqz" pod "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" (UID: "2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6"). InnerVolumeSpecName "kube-api-access-lgzqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:32.436126 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.436050    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-podinfo" pod "59ed629b-1c29-4cfc-909c-7b4089c141ac" (UID: "59ed629b-1c29-4cfc-909c-7b4089c141ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:32.436206 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.436132    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-envoy" pod "59ed629b-1c29-4cfc-909c-7b4089c141ac" (UID: "59ed629b-1c29-4cfc-909c-7b4089c141ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.436206 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.436151    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-token" pod "e71caf86-9d04-4442-bcfd-5b7bf21900e6" (UID: "e71caf86-9d04-4442-bcfd-5b7bf21900e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:32.436206 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.436134    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-kube-api-access-4ffnv" pod "59ed629b-1c29-4cfc-909c-7b4089c141ac" (UID: "59ed629b-1c29-4cfc-909c-7b4089c141ac"). InnerVolumeSpecName "kube-api-access-4ffnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:32.436451 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.436431    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-token" pod "59ed629b-1c29-4cfc-909c-7b4089c141ac" (UID: "59ed629b-1c29-4cfc-909c-7b4089c141ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:32.436545 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.436528    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-envoy" pod "e71caf86-9d04-4442-bcfd-5b7bf21900e6" (UID: "e71caf86-9d04-4442-bcfd-5b7bf21900e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:32.534920 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.534881    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ffnv\" (UniqueName: \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-kube-api-access-4ffnv\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.534920 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.534920    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.534937    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.534951    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.534965    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.534979    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.535011    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71caf86-9d04-4442-bcfd-5b7bf21900e6-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.535027    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g64rv\" (UniqueName: \"kubernetes.io/projected/e71caf86-9d04-4442-bcfd-5b7bf21900e6-kube-api-access-g64rv\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.535040    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.535054    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgzqz\" (UniqueName: \"kubernetes.io/projected/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-kube-api-access-lgzqz\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.535069    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ed629b-1c29-4cfc-909c-7b4089c141ac-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.535084    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e71caf86-9d04-4442-bcfd-5b7bf21900e6-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.535097    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.535169 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.535110    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:32.711976 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.711888    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:33:32.712154 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.711964    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:33:32.996729 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:32.996670    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:33:33.017142 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.017117    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:33:33.033206 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.033178    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:33:33.040126 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040101    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-credential-socket\") pod \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") "
Nov 02 22:33:33.040242 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040185    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-data\") pod \"5d7463af-b57c-45a2-a286-b7a70fadb525\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") "
Nov 02 22:33:33.040242 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040226    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-data\") pod \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040269    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d7463af-b57c-45a2-a286-b7a70fadb525-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d7463af-b57c-45a2-a286-b7a70fadb525-istiod-ca-cert\") pod \"5d7463af-b57c-45a2-a286-b7a70fadb525\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040327    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-envoy\") pod \"5d7463af-b57c-45a2-a286-b7a70fadb525\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040392    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-token\" (UniqueName: \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-token\") pod \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040420    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-credential-socket" pod "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" (UID: "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040445    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-kube-api-access-hkn5l\" (UniqueName: \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-kube-api-access-hkn5l\") pod \"5d7463af-b57c-45a2-a286-b7a70fadb525\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040521    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-data" pod "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" (UID: "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040538    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-kube-api-access-nbz7b\" (UniqueName: \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-kube-api-access-nbz7b\") pod \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040523    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-data" pod "5d7463af-b57c-45a2-a286-b7a70fadb525" (UID: "5d7463af-b57c-45a2-a286-b7a70fadb525"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040567    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-socket\") pod \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040597    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-certs\") pod \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040628    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-envoy\") pod \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040666    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-certs\") pod \"5d7463af-b57c-45a2-a286-b7a70fadb525\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040691    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-token\" (UniqueName: \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-token\") pod \"5d7463af-b57c-45a2-a286-b7a70fadb525\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") "
Nov 02 22:33:33.040777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040749    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-podinfo\") pod \"5d7463af-b57c-45a2-a286-b7a70fadb525\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") "
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040793    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istiod-ca-cert\") pod \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") "
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040797    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-socket" pod "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" (UID: "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040718    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d7463af-b57c-45a2-a286-b7a70fadb525-istiod-ca-cert" pod "5d7463af-b57c-45a2-a286-b7a70fadb525" (UID: "5d7463af-b57c-45a2-a286-b7a70fadb525"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040832    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-credential-socket\") pod \"5d7463af-b57c-45a2-a286-b7a70fadb525\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") "
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040879    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-socket\") pod \"5d7463af-b57c-45a2-a286-b7a70fadb525\" (UID: \"5d7463af-b57c-45a2-a286-b7a70fadb525\") "
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.040939    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-podinfo\") pod \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\" (UID: \"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04\") "
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041011    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-certs" pod "5d7463af-b57c-45a2-a286-b7a70fadb525" (UID: "5d7463af-b57c-45a2-a286-b7a70fadb525"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041020    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-certs" pod "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" (UID: "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041218    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-credential-socket" pod "5d7463af-b57c-45a2-a286-b7a70fadb525" (UID: "5d7463af-b57c-45a2-a286-b7a70fadb525"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041535    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041556    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041572    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041588    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d7463af-b57c-45a2-a286-b7a70fadb525-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.041607 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041603    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.042324 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041616    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.042324 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041631    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.042324 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041645    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.042324 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041799    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-socket" pod "5d7463af-b57c-45a2-a286-b7a70fadb525" (UID: "5d7463af-b57c-45a2-a286-b7a70fadb525"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.042324 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.041953    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istiod-ca-cert" pod "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" (UID: "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:33.043194 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.043157    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-kube-api-access-hkn5l" pod "5d7463af-b57c-45a2-a286-b7a70fadb525" (UID: "5d7463af-b57c-45a2-a286-b7a70fadb525"). InnerVolumeSpecName "kube-api-access-hkn5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:33.043292 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.043274    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-kube-api-access-nbz7b" pod "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" (UID: "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04"). InnerVolumeSpecName "kube-api-access-nbz7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:33.043635 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.043587    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-podinfo" pod "5d7463af-b57c-45a2-a286-b7a70fadb525" (UID: "5d7463af-b57c-45a2-a286-b7a70fadb525"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:33.043858 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.043837    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-podinfo" pod "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" (UID: "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:33.043957 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.043862    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-token" pod "5d7463af-b57c-45a2-a286-b7a70fadb525" (UID: "5d7463af-b57c-45a2-a286-b7a70fadb525"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:33.043957 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.043931    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-token" pod "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" (UID: "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:33.044084 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.044069    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-envoy" pod "5d7463af-b57c-45a2-a286-b7a70fadb525" (UID: "5d7463af-b57c-45a2-a286-b7a70fadb525"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.044495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.044475    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-envoy" pod "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" (UID: "69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.132954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.132915    5067 generic.go:356] "Generic (PLEG): container finished" podID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerID="b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5" exitCode=0
Nov 02 22:33:33.133472 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.132995    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"
Nov 02 22:33:33.133472 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.133011    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" event={"ID":"f4bc6e5d-e119-42e0-8254-df6772091d7e","Type":"ContainerDied","Data":"b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5"}
Nov 02 22:33:33.133472 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.133087    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv" event={"ID":"f4bc6e5d-e119-42e0-8254-df6772091d7e","Type":"ContainerDied","Data":"a09266e98dd9a80ebc1b73504515113a37a8ea91dd5721176d64e031ab5e701f"}
Nov 02 22:33:33.133472 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.133121    5067 scope.go:122] "RemoveContainer" containerID="b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5"
Nov 02 22:33:33.134790 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.134758    5067 generic.go:356] "Generic (PLEG): container finished" podID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerID="f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397" exitCode=0
Nov 02 22:33:33.134888 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.134809    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"
Nov 02 22:33:33.134942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.134908    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" event={"ID":"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04","Type":"ContainerDied","Data":"f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397"}
Nov 02 22:33:33.135003 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.134949    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt" event={"ID":"69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04","Type":"ContainerDied","Data":"a18369ba026b780a2193baad454bc90cf00e28d629dad39a8e4ed5f5ec544485"}
Nov 02 22:33:33.136709 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.136685    5067 generic.go:356] "Generic (PLEG): container finished" podID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerID="11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2" exitCode=0
Nov 02 22:33:33.136832 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.136818    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"
Nov 02 22:33:33.137287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.137262    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" event={"ID":"e71caf86-9d04-4442-bcfd-5b7bf21900e6","Type":"ContainerDied","Data":"11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2"}
Nov 02 22:33:33.137415 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.137318    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt" event={"ID":"e71caf86-9d04-4442-bcfd-5b7bf21900e6","Type":"ContainerDied","Data":"c249770cc1d1c0713b0afec2c113cb3ab9be954d6895097eb9771c6943a4b925"}
Nov 02 22:33:33.142162 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.142130    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-podinfo\") pod \"f4bc6e5d-e119-42e0-8254-df6772091d7e\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") "
Nov 02 22:33:33.142278 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.142185    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-envoy\") pod \"f4bc6e5d-e119-42e0-8254-df6772091d7e\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") "
Nov 02 22:33:33.142278 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.142265    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4bc6e5d-e119-42e0-8254-df6772091d7e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4bc6e5d-e119-42e0-8254-df6772091d7e-istiod-ca-cert\") pod \"f4bc6e5d-e119-42e0-8254-df6772091d7e\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") "
Nov 02 22:33:33.142422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.142292    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-token\" (UniqueName: \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-token\") pod \"f4bc6e5d-e119-42e0-8254-df6772091d7e\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") "
Nov 02 22:33:33.142422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.142404    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-certs\") pod \"f4bc6e5d-e119-42e0-8254-df6772091d7e\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") "
Nov 02 22:33:33.142730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.142469    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-socket\") pod \"f4bc6e5d-e119-42e0-8254-df6772091d7e\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") "
Nov 02 22:33:33.142730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.142519    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-kube-api-access-vlgcp\" (UniqueName: \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-kube-api-access-vlgcp\") pod \"f4bc6e5d-e119-42e0-8254-df6772091d7e\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") "
Nov 02 22:33:33.142730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.142583    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-credential-socket\") pod \"f4bc6e5d-e119-42e0-8254-df6772091d7e\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") "
Nov 02 22:33:33.142730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.142614    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-data\") pod \"f4bc6e5d-e119-42e0-8254-df6772091d7e\" (UID: \"f4bc6e5d-e119-42e0-8254-df6772091d7e\") "
Nov 02 22:33:33.143082 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143064    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbz7b\" (UniqueName: \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-kube-api-access-nbz7b\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.143135 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143087    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.143135 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143101    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.143135 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143115    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.143135 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143129    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.143328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143142    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.143328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143155    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.143328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143168    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d7463af-b57c-45a2-a286-b7a70fadb525-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.143328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143197    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.143328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143212    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkn5l\" (UniqueName: \"kubernetes.io/projected/5d7463af-b57c-45a2-a286-b7a70fadb525-kube-api-access-hkn5l\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.143612 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.143458    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-data" pod "f4bc6e5d-e119-42e0-8254-df6772091d7e" (UID: "f4bc6e5d-e119-42e0-8254-df6772091d7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.144644 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.144320    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-credential-socket" pod "f4bc6e5d-e119-42e0-8254-df6772091d7e" (UID: "f4bc6e5d-e119-42e0-8254-df6772091d7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.144644 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.144492    5067 generic.go:356] "Generic (PLEG): container finished" podID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerID="94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06" exitCode=0
Nov 02 22:33:33.144644 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.144532    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-certs" pod "f4bc6e5d-e119-42e0-8254-df6772091d7e" (UID: "f4bc6e5d-e119-42e0-8254-df6772091d7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.144644 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.144609    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" event={"ID":"59ed629b-1c29-4cfc-909c-7b4089c141ac","Type":"ContainerDied","Data":"94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06"}
Nov 02 22:33:33.144901 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.144661    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s" event={"ID":"59ed629b-1c29-4cfc-909c-7b4089c141ac","Type":"ContainerDied","Data":"f20e485be5057e7bb0336c29e3b01e6d4997fb74a1d260b9a056c69741a54c07"}
Nov 02 22:33:33.144901 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.144619    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"
Nov 02 22:33:33.149405 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.146886    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-socket" pod "f4bc6e5d-e119-42e0-8254-df6772091d7e" (UID: "f4bc6e5d-e119-42e0-8254-df6772091d7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.149405 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.147216    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4bc6e5d-e119-42e0-8254-df6772091d7e-istiod-ca-cert" pod "f4bc6e5d-e119-42e0-8254-df6772091d7e" (UID: "f4bc6e5d-e119-42e0-8254-df6772091d7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:33.150813 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.150485    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-envoy" pod "f4bc6e5d-e119-42e0-8254-df6772091d7e" (UID: "f4bc6e5d-e119-42e0-8254-df6772091d7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:33.150813 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.150554    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-kube-api-access-vlgcp" pod "f4bc6e5d-e119-42e0-8254-df6772091d7e" (UID: "f4bc6e5d-e119-42e0-8254-df6772091d7e"). InnerVolumeSpecName "kube-api-access-vlgcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:33.150813 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.150616    5067 scope.go:122] "RemoveContainer" containerID="c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021"
Nov 02 22:33:33.150813 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.150704    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-token" pod "f4bc6e5d-e119-42e0-8254-df6772091d7e" (UID: "f4bc6e5d-e119-42e0-8254-df6772091d7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:33.150813 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.150721    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-podinfo" pod "f4bc6e5d-e119-42e0-8254-df6772091d7e" (UID: "f4bc6e5d-e119-42e0-8254-df6772091d7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:33.153708 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.153678    5067 generic.go:356] "Generic (PLEG): container finished" podID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerID="d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa" exitCode=0
Nov 02 22:33:33.153810 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.153734    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"
Nov 02 22:33:33.153810 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.153766    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" event={"ID":"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6","Type":"ContainerDied","Data":"d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa"}
Nov 02 22:33:33.153810 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.153805    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4" event={"ID":"2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6","Type":"ContainerDied","Data":"5532528ca92e02c05ead91e66f38d5d96e65c0943a08d6fbb18742bc17601288"}
Nov 02 22:33:33.155425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.155401    5067 generic.go:356] "Generic (PLEG): container finished" podID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerID="e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b" exitCode=0
Nov 02 22:33:33.155526 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.155477    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" event={"ID":"5d7463af-b57c-45a2-a286-b7a70fadb525","Type":"ContainerDied","Data":"e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b"}
Nov 02 22:33:33.155526 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.155513    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc" event={"ID":"5d7463af-b57c-45a2-a286-b7a70fadb525","Type":"ContainerDied","Data":"a8c1d7c85ab559c01b32da4d3947d8578ab0ab7e2ba46b1650b1f4063e178d96"}
Nov 02 22:33:33.155701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.155576    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"
Nov 02 22:33:33.179182 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.179119    5067 scope.go:122] "RemoveContainer" containerID="b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5"
Nov 02 22:33:33.179533 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:33.179473    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5\": container with ID starting with b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5 not found: ID does not exist" containerID="b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5"
Nov 02 22:33:33.179533 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.179509    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5"} err="failed to get container status \"b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5\": rpc error: code = NotFound desc = could not find container \"b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5\": container with ID starting with b4e2df949d5dc922fa169879f56a1c201789435a6f79e80962b2c9cbe4d826c5 not found: ID does not exist"
Nov 02 22:33:33.179533 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.179528    5067 scope.go:122] "RemoveContainer" containerID="c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021"
Nov 02 22:33:33.179835 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:33.179806    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021\": container with ID starting with c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021 not found: ID does not exist" containerID="c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021"
Nov 02 22:33:33.179880 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.179842    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021"} err="failed to get container status \"c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021\": rpc error: code = NotFound desc = could not find container \"c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021\": container with ID starting with c85d5ea7785bc8fc34d2c34a8d0ea04f2c66d32812b274177ce584d1a4865021 not found: ID does not exist"
Nov 02 22:33:33.179880 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.179859    5067 scope.go:122] "RemoveContainer" containerID="f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397"
Nov 02 22:33:33.190212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.190179    5067 scope.go:122] "RemoveContainer" containerID="f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397"
Nov 02 22:33:33.190786 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:33.190754    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397\": container with ID starting with f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397 not found: ID does not exist" containerID="f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397"
Nov 02 22:33:33.190899 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.190796    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397"} err="failed to get container status \"f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397\": rpc error: code = NotFound desc = could not find container \"f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397\": container with ID starting with f7cf0550d3114814fe4cb02ae7725a26f3256698fe6756bc4ba1e16b501b1397 not found: ID does not exist"
Nov 02 22:33:33.190899 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.190822    5067 scope.go:122] "RemoveContainer" containerID="11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2"
Nov 02 22:33:33.203781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.203754    5067 scope.go:122] "RemoveContainer" containerID="11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2"
Nov 02 22:33:33.204152 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:33.204127    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2\": container with ID starting with 11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2 not found: ID does not exist" containerID="11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2"
Nov 02 22:33:33.204245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.204159    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2"} err="failed to get container status \"11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2\": rpc error: code = NotFound desc = could not find container \"11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2\": container with ID starting with 11afe9e88eed4abb4c0384cfe18fe0b512638d024ae780ff0096b081374127c2 not found: ID does not exist"
Nov 02 22:33:33.204245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.204183    5067 scope.go:122] "RemoveContainer" containerID="94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06"
Nov 02 22:33:33.213831 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.213809    5067 scope.go:122] "RemoveContainer" containerID="94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06"
Nov 02 22:33:33.214189 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:33.214162    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06\": container with ID starting with 94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06 not found: ID does not exist" containerID="94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06"
Nov 02 22:33:33.214306 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.214199    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06"} err="failed to get container status \"94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06\": rpc error: code = NotFound desc = could not find container \"94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06\": container with ID starting with 94f52ad97aafefe95a56dc7d82e9172ea2a24912fc2c9e3dc7c4255d86005b06 not found: ID does not exist"
Nov 02 22:33:33.214306 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.214220    5067 scope.go:122] "RemoveContainer" containerID="d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa"
Nov 02 22:33:33.223444 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.223272    5067 scope.go:122] "RemoveContainer" containerID="d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa"
Nov 02 22:33:33.223963 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:33.223936    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa\": container with ID starting with d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa not found: ID does not exist" containerID="d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa"
Nov 02 22:33:33.224055 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.223973    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa"} err="failed to get container status \"d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa\": rpc error: code = NotFound desc = could not find container \"d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa\": container with ID starting with d2d750cc2bc75c00268b560fa47e9c529f13d224233379e506342046a2200afa not found: ID does not exist"
Nov 02 22:33:33.224055 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.223997    5067 scope.go:122] "RemoveContainer" containerID="e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b"
Nov 02 22:33:33.237252 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.237229    5067 scope.go:122] "RemoveContainer" containerID="e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b"
Nov 02 22:33:33.237836 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:33.237771    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b\": container with ID starting with e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b not found: ID does not exist" containerID="e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b"
Nov 02 22:33:33.237836 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.237806    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b"} err="failed to get container status \"e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b\": rpc error: code = NotFound desc = could not find container \"e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b\": container with ID starting with e4330bed63535557b29fea21084b09c2892d7085bda635c1be70788d4b9fb08b not found: ID does not exist"
Nov 02 22:33:33.244839 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.244814    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.244954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.244841    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlgcp\" (UniqueName: \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-kube-api-access-vlgcp\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.244954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.244856    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.244954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.244868    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.244954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.244882    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.244954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.244895    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.244954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.244912    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4bc6e5d-e119-42e0-8254-df6772091d7e-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.244954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.244926    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4bc6e5d-e119-42e0-8254-df6772091d7e-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.244954 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.244938    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4bc6e5d-e119-42e0-8254-df6772091d7e-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:33.443499 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.443470    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"]
Nov 02 22:33:33.448909 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.448883    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-vdtmt"]
Nov 02 22:33:33.552646 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.552611    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"]
Nov 02 22:33:33.556962 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.556927    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-6wbtt"]
Nov 02 22:33:33.708289 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.708252    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"]
Nov 02 22:33:33.713495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.713468    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wq26s"]
Nov 02 22:33:33.813409 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.813307    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"]
Nov 02 22:33:33.817883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.817853    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-49a557-openshift-default-5796bb9575-wlfm4"]
Nov 02 22:33:33.844781 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.844752    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" path="/var/lib/kubelet/pods/2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6/volumes"
Nov 02 22:33:33.845262 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.845248    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" path="/var/lib/kubelet/pods/59ed629b-1c29-4cfc-909c-7b4089c141ac/volumes"
Nov 02 22:33:33.845706 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.845692    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" path="/var/lib/kubelet/pods/69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04/volumes"
Nov 02 22:33:33.846424 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.846410    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" path="/var/lib/kubelet/pods/e71caf86-9d04-4442-bcfd-5b7bf21900e6/volumes"
Nov 02 22:33:33.919416 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.919382    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"]
Nov 02 22:33:33.923649 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:33.923619    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-44gjc"]
Nov 02 22:33:34.035711 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:34.035668    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"]
Nov 02 22:33:34.040188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:34.040146    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-98a222-openshift-default-587d758db7-twpzv"]
Nov 02 22:33:34.810925 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:34.810885    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:33:34.811088 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:34.810946    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:33:34.914910 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:34.914868    5067 patch_prober.go:29] interesting pod/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:33:34.915065 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:34.914941    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:33:35.807145 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.807075    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:33:35.845655 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.845620    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" path="/var/lib/kubelet/pods/5d7463af-b57c-45a2-a286-b7a70fadb525/volumes"
Nov 02 22:33:35.846072 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.846058    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" path="/var/lib/kubelet/pods/f4bc6e5d-e119-42e0-8254-df6772091d7e/volumes"
Nov 02 22:33:35.874683 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.874650    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-token\" (UniqueName: \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-token\") pod \"06409ed1-ce32-4d21-b844-b18b32615f3d\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") "
Nov 02 22:33:35.874830 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.874721    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06409ed1-ce32-4d21-b844-b18b32615f3d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06409ed1-ce32-4d21-b844-b18b32615f3d-istiod-ca-cert\") pod \"06409ed1-ce32-4d21-b844-b18b32615f3d\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") "
Nov 02 22:33:35.874830 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.874795    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-podinfo\") pod \"06409ed1-ce32-4d21-b844-b18b32615f3d\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") "
Nov 02 22:33:35.874969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.874843    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-credential-socket\") pod \"06409ed1-ce32-4d21-b844-b18b32615f3d\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") "
Nov 02 22:33:35.874969 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.874876    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-data\") pod \"06409ed1-ce32-4d21-b844-b18b32615f3d\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") "
Nov 02 22:33:35.875073 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.874969    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-kube-api-access-p5dq9\" (UniqueName: \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-kube-api-access-p5dq9\") pod \"06409ed1-ce32-4d21-b844-b18b32615f3d\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") "
Nov 02 22:33:35.875073 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875007    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-certs\") pod \"06409ed1-ce32-4d21-b844-b18b32615f3d\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") "
Nov 02 22:33:35.875178 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875100    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-socket\") pod \"06409ed1-ce32-4d21-b844-b18b32615f3d\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") "
Nov 02 22:33:35.875178 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875127    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-credential-socket" pod "06409ed1-ce32-4d21-b844-b18b32615f3d" (UID: "06409ed1-ce32-4d21-b844-b18b32615f3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:35.875178 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875135    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-envoy\") pod \"06409ed1-ce32-4d21-b844-b18b32615f3d\" (UID: \"06409ed1-ce32-4d21-b844-b18b32615f3d\") "
Nov 02 22:33:35.875334 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875224    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-data" pod "06409ed1-ce32-4d21-b844-b18b32615f3d" (UID: "06409ed1-ce32-4d21-b844-b18b32615f3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:35.875334 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875252    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06409ed1-ce32-4d21-b844-b18b32615f3d-istiod-ca-cert" pod "06409ed1-ce32-4d21-b844-b18b32615f3d" (UID: "06409ed1-ce32-4d21-b844-b18b32615f3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:35.875334 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875289    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-certs" pod "06409ed1-ce32-4d21-b844-b18b32615f3d" (UID: "06409ed1-ce32-4d21-b844-b18b32615f3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:35.875546 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875449    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-socket" pod "06409ed1-ce32-4d21-b844-b18b32615f3d" (UID: "06409ed1-ce32-4d21-b844-b18b32615f3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:35.875656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875639    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:35.875716 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875658    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:35.875716 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875672    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:35.875716 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875683    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:35.875716 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.875698    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06409ed1-ce32-4d21-b844-b18b32615f3d-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:35.877109 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.877085    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-podinfo" pod "06409ed1-ce32-4d21-b844-b18b32615f3d" (UID: "06409ed1-ce32-4d21-b844-b18b32615f3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:35.877203 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.877180    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-kube-api-access-p5dq9" pod "06409ed1-ce32-4d21-b844-b18b32615f3d" (UID: "06409ed1-ce32-4d21-b844-b18b32615f3d"). InnerVolumeSpecName "kube-api-access-p5dq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:35.877431 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.877405    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-envoy" pod "06409ed1-ce32-4d21-b844-b18b32615f3d" (UID: "06409ed1-ce32-4d21-b844-b18b32615f3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:35.877537 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.877411    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-token" pod "06409ed1-ce32-4d21-b844-b18b32615f3d" (UID: "06409ed1-ce32-4d21-b844-b18b32615f3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:35.977047 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.977006    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:35.977047 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.977043    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5dq9\" (UniqueName: \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-kube-api-access-p5dq9\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:35.977047 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.977055    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:35.977292 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:35.977063    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06409ed1-ce32-4d21-b844-b18b32615f3d-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.179180 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.179053    5067 generic.go:356] "Generic (PLEG): container finished" podID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerID="3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52" exitCode=0
Nov 02 22:33:36.179180 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.179164    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"
Nov 02 22:33:36.179555 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.179457    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" event={"ID":"06409ed1-ce32-4d21-b844-b18b32615f3d","Type":"ContainerDied","Data":"3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52"}
Nov 02 22:33:36.179555 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.179542    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx" event={"ID":"06409ed1-ce32-4d21-b844-b18b32615f3d","Type":"ContainerDied","Data":"457202471b23525f00ace1b324c3967d34c35f95f88ee4cfd72db2ee61ed1e2d"}
Nov 02 22:33:36.179717 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.179577    5067 scope.go:122] "RemoveContainer" containerID="3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52"
Nov 02 22:33:36.197974 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.197931    5067 generic.go:356] "Generic (PLEG): container finished" podID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerID="14da73a16794fd9cb0a6e1f33bd7e15e69d03d55a040b10e2af573d382662852" exitCode=0
Nov 02 22:33:36.198768 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.198055    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" event={"ID":"deca162c-c1be-4048-89f4-1ad60fd1ee87","Type":"ContainerDied","Data":"14da73a16794fd9cb0a6e1f33bd7e15e69d03d55a040b10e2af573d382662852"}
Nov 02 22:33:36.200201 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.200174    5067 generic.go:356] "Generic (PLEG): container finished" podID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerID="29a6bf44b255c7af1371bddd91f93f3dbae162f323070dc2e0c53c20a6c7f547" exitCode=0
Nov 02 22:33:36.200297 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.200251    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" event={"ID":"7326e5f6-1330-453b-99a0-1eae9a0c89fc","Type":"ContainerDied","Data":"29a6bf44b255c7af1371bddd91f93f3dbae162f323070dc2e0c53c20a6c7f547"}
Nov 02 22:33:36.214538 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.214513    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:33:36.215013 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.214930    5067 scope.go:122] "RemoveContainer" containerID="3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52"
Nov 02 22:33:36.215396 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:36.215297    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52\": container with ID starting with 3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52 not found: ID does not exist" containerID="3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52"
Nov 02 22:33:36.215396 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.215335    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52"} err="failed to get container status \"3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52\": rpc error: code = NotFound desc = could not find container \"3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52\": container with ID starting with 3548655d1a86f1202af6ebb835a6be71b968e6bc43d99ff8dc9d01d691c96d52 not found: ID does not exist"
Nov 02 22:33:36.224923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.224896    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:33:36.279662 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.279629    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/deca162c-c1be-4048-89f4-1ad60fd1ee87-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deca162c-c1be-4048-89f4-1ad60fd1ee87-istiod-ca-cert\") pod \"deca162c-c1be-4048-89f4-1ad60fd1ee87\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") "
Nov 02 22:33:36.279851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.279676    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-kube-api-access-fm625\" (UniqueName: \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-kube-api-access-fm625\") pod \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") "
Nov 02 22:33:36.279851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.279713    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-envoy\") pod \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") "
Nov 02 22:33:36.279970 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.279876    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-data\") pod \"deca162c-c1be-4048-89f4-1ad60fd1ee87\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") "
Nov 02 22:33:36.279970 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.279928    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-socket\") pod \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") "
Nov 02 22:33:36.279970 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.279950    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-podinfo\") pod \"deca162c-c1be-4048-89f4-1ad60fd1ee87\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") "
Nov 02 22:33:36.280131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.279976    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-podinfo\") pod \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") "
Nov 02 22:33:36.280131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280010    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-certs\") pod \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") "
Nov 02 22:33:36.280131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280043    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-token\" (UniqueName: \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-token\") pod \"deca162c-c1be-4048-89f4-1ad60fd1ee87\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") "
Nov 02 22:33:36.280131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280073    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-envoy\") pod \"deca162c-c1be-4048-89f4-1ad60fd1ee87\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") "
Nov 02 22:33:36.280131 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280105    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-socket\") pod \"deca162c-c1be-4048-89f4-1ad60fd1ee87\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") "
Nov 02 22:33:36.280389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280152    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-kube-api-access-244tp\" (UniqueName: \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-kube-api-access-244tp\") pod \"deca162c-c1be-4048-89f4-1ad60fd1ee87\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") "
Nov 02 22:33:36.280389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280071    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deca162c-c1be-4048-89f4-1ad60fd1ee87-istiod-ca-cert" pod "deca162c-c1be-4048-89f4-1ad60fd1ee87" (UID: "deca162c-c1be-4048-89f4-1ad60fd1ee87"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:36.280389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280189    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-socket" pod "7326e5f6-1330-453b-99a0-1eae9a0c89fc" (UID: "7326e5f6-1330-453b-99a0-1eae9a0c89fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:36.280389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280201    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-data" pod "deca162c-c1be-4048-89f4-1ad60fd1ee87" (UID: "deca162c-c1be-4048-89f4-1ad60fd1ee87"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:36.280389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280252    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-certs\") pod \"deca162c-c1be-4048-89f4-1ad60fd1ee87\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") "
Nov 02 22:33:36.280389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280296    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-token\" (UniqueName: \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-token\") pod \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") "
Nov 02 22:33:36.280389 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280331    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-credential-socket\") pod \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") "
Nov 02 22:33:36.280753 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280439    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istiod-ca-cert\") pod \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") "
Nov 02 22:33:36.280753 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280514    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-data\") pod \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\" (UID: \"7326e5f6-1330-453b-99a0-1eae9a0c89fc\") "
Nov 02 22:33:36.280753 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280546    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-credential-socket\") pod \"deca162c-c1be-4048-89f4-1ad60fd1ee87\" (UID: \"deca162c-c1be-4048-89f4-1ad60fd1ee87\") "
Nov 02 22:33:36.280753 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280441    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-certs" pod "7326e5f6-1330-453b-99a0-1eae9a0c89fc" (UID: "7326e5f6-1330-453b-99a0-1eae9a0c89fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:36.280970 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280810    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-socket" pod "deca162c-c1be-4048-89f4-1ad60fd1ee87" (UID: "deca162c-c1be-4048-89f4-1ad60fd1ee87"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:36.280970 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280942    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-credential-socket" pod "deca162c-c1be-4048-89f4-1ad60fd1ee87" (UID: "deca162c-c1be-4048-89f4-1ad60fd1ee87"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:36.280970 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.280647    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-certs" pod "deca162c-c1be-4048-89f4-1ad60fd1ee87" (UID: "deca162c-c1be-4048-89f4-1ad60fd1ee87"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:36.281205 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.281044    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istiod-ca-cert" pod "7326e5f6-1330-453b-99a0-1eae9a0c89fc" (UID: "7326e5f6-1330-453b-99a0-1eae9a0c89fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:33:36.281205 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.281162    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.281205 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.281185    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.281205 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.281202    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deca162c-c1be-4048-89f4-1ad60fd1ee87-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.281465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.281217    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.281465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.281230    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.281465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.281244    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.281465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.281259    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.281465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.281182    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-data" pod "7326e5f6-1330-453b-99a0-1eae9a0c89fc" (UID: "7326e5f6-1330-453b-99a0-1eae9a0c89fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:36.281465 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.281400    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-credential-socket" pod "7326e5f6-1330-453b-99a0-1eae9a0c89fc" (UID: "7326e5f6-1330-453b-99a0-1eae9a0c89fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:36.282506 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.282477    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-kube-api-access-fm625" pod "7326e5f6-1330-453b-99a0-1eae9a0c89fc" (UID: "7326e5f6-1330-453b-99a0-1eae9a0c89fc"). InnerVolumeSpecName "kube-api-access-fm625". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:36.282892 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.282868    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-envoy" pod "deca162c-c1be-4048-89f4-1ad60fd1ee87" (UID: "deca162c-c1be-4048-89f4-1ad60fd1ee87"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:36.283165 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.283135    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-envoy" pod "7326e5f6-1330-453b-99a0-1eae9a0c89fc" (UID: "7326e5f6-1330-453b-99a0-1eae9a0c89fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:33:36.283264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.283163    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-kube-api-access-244tp" pod "deca162c-c1be-4048-89f4-1ad60fd1ee87" (UID: "deca162c-c1be-4048-89f4-1ad60fd1ee87"). InnerVolumeSpecName "kube-api-access-244tp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:36.283264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.283196    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-podinfo" pod "deca162c-c1be-4048-89f4-1ad60fd1ee87" (UID: "deca162c-c1be-4048-89f4-1ad60fd1ee87"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:36.283264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.283205    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-token" pod "deca162c-c1be-4048-89f4-1ad60fd1ee87" (UID: "deca162c-c1be-4048-89f4-1ad60fd1ee87"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:36.283429 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.283408    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-podinfo" pod "7326e5f6-1330-453b-99a0-1eae9a0c89fc" (UID: "7326e5f6-1330-453b-99a0-1eae9a0c89fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:33:36.284198 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.284182    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-token" pod "7326e5f6-1330-453b-99a0-1eae9a0c89fc" (UID: "7326e5f6-1330-453b-99a0-1eae9a0c89fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:33:36.336059 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.336022    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"]
Nov 02 22:33:36.340515 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.340485    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-nl7zx"]
Nov 02 22:33:36.382254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382179    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.382254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382205    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.382254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382217    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-244tp\" (UniqueName: \"kubernetes.io/projected/deca162c-c1be-4048-89f4-1ad60fd1ee87-kube-api-access-244tp\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.382254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382228    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.382254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382237    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.382254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382246    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.382254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382253    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.382618 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382262    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm625\" (UniqueName: \"kubernetes.io/projected/7326e5f6-1330-453b-99a0-1eae9a0c89fc-kube-api-access-fm625\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.382618 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382271    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.382618 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382279    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deca162c-c1be-4048-89f4-1ad60fd1ee87-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:36.382618 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:36.382288    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7326e5f6-1330-453b-99a0-1eae9a0c89fc-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:33:37.208424 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.208396    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"
Nov 02 22:33:37.208938 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.208391    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5" event={"ID":"deca162c-c1be-4048-89f4-1ad60fd1ee87","Type":"ContainerDied","Data":"0032f7089c3231902690a9eb951c310e862c7296f23c09aebfddcd33e201396e"}
Nov 02 22:33:37.208938 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.208535    5067 scope.go:122] "RemoveContainer" containerID="14da73a16794fd9cb0a6e1f33bd7e15e69d03d55a040b10e2af573d382662852"
Nov 02 22:33:37.209933 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.209917    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"
Nov 02 22:33:37.210019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.209963    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl" event={"ID":"7326e5f6-1330-453b-99a0-1eae9a0c89fc","Type":"ContainerDied","Data":"791967c8f19d6c590def92f77190dc9db033576e4c30773eb8e099e99351aa24"}
Nov 02 22:33:37.219410 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.219388    5067 scope.go:122] "RemoveContainer" containerID="29a6bf44b255c7af1371bddd91f93f3dbae162f323070dc2e0c53c20a6c7f547"
Nov 02 22:33:37.363173 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.363137    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"]
Nov 02 22:33:37.409805 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.409767    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-5gsc5"]
Nov 02 22:33:37.469758 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.469723    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"]
Nov 02 22:33:37.474014 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.473988    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-fe810f-openshift-default-86c89d8ff9-hblhl"]
Nov 02 22:33:37.575437 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:33:37.575403    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:33:37.845223 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.845144    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" path="/var/lib/kubelet/pods/06409ed1-ce32-4d21-b844-b18b32615f3d/volumes"
Nov 02 22:33:37.845665 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.845651    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" path="/var/lib/kubelet/pods/7326e5f6-1330-453b-99a0-1eae9a0c89fc/volumes"
Nov 02 22:33:37.846057 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:33:37.846045    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" path="/var/lib/kubelet/pods/deca162c-c1be-4048-89f4-1ad60fd1ee87/volumes"
Nov 02 22:34:07.579680 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:34:07.579633    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:34:24.456184 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:24.456147    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"]
Nov 02 22:34:24.456610 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:24.456409    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy" containerID="cri-o://67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f" gracePeriod=30
Nov 02 22:34:28.772659 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:28.772628    5067 patch_prober.go:29] interesting pod/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 22:34:28.772961 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:28.772715    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 22:34:29.716511 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.716485    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:34:29.883708 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.883622    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istiod-ca-cert\") pod \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") "
Nov 02 22:34:29.884164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.883742    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-token\" (UniqueName: \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-token\") pod \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") "
Nov 02 22:34:29.884164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.883777    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-podinfo\") pod \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") "
Nov 02 22:34:29.884164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.883807    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-certs\") pod \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") "
Nov 02 22:34:29.884164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.883860    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-kube-api-access-cgfm2\" (UniqueName: \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-kube-api-access-cgfm2\") pod \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") "
Nov 02 22:34:29.884164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.883917    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-socket\") pod \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") "
Nov 02 22:34:29.884164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.883959    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-envoy\") pod \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") "
Nov 02 22:34:29.884164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.883987    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-credential-socket\") pod \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") "
Nov 02 22:34:29.884164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.884021    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-data\") pod \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\" (UID: \"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01\") "
Nov 02 22:34:29.884164 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.884030    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istiod-ca-cert" pod "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" (UID: "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:34:29.884685 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.884437    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:29.884685 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.884456    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-credential-socket" pod "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" (UID: "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:29.884685 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.884495    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-certs" pod "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" (UID: "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:29.884685 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.884564    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-socket" pod "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" (UID: "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:29.884685 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.884658    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-data" pod "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" (UID: "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:29.886947 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.886906    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-podinfo" pod "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" (UID: "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:34:29.887084 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.886939    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-token" pod "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" (UID: "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:34:29.887199 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.887174    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-envoy" pod "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" (UID: "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:29.887313 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.887209    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-kube-api-access-cgfm2" pod "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" (UID: "7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01"). InnerVolumeSpecName "kube-api-access-cgfm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:34:29.926751 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.926712    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"]
Nov 02 22:34:29.928952 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.928920    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"]
Nov 02 22:34:29.985777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.985615    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:29.985777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.985651    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:29.985777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.985668    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:29.985777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.985683    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:29.985777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.985697    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:29.985777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.985711    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:29.985777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.985727    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgfm2\" (UniqueName: \"kubernetes.io/projected/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-kube-api-access-cgfm2\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:29.985777 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:29.985745    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:30.537419 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.537351    5067 generic.go:356] "Generic (PLEG): container finished" podID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerID="67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f" exitCode=0
Nov 02 22:34:30.537419 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.537395    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" event={"ID":"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01","Type":"ContainerDied","Data":"67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f"}
Nov 02 22:34:30.537663 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.537445    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"
Nov 02 22:34:30.537663 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.537461    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt" event={"ID":"7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01","Type":"ContainerDied","Data":"e5f381c7fdee6b54583f9bf1bef91fecde170a5962ccab9d621e7e94c1ce5bfc"}
Nov 02 22:34:30.537663 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.537492    5067 scope.go:122] "RemoveContainer" containerID="67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f"
Nov 02 22:34:30.548500 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.548479    5067 scope.go:122] "RemoveContainer" containerID="aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6"
Nov 02 22:34:30.556800 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.556782    5067 scope.go:122] "RemoveContainer" containerID="67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f"
Nov 02 22:34:30.557067 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:34:30.557050    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f\": container with ID starting with 67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f not found: ID does not exist" containerID="67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f"
Nov 02 22:34:30.557121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.557078    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f"} err="failed to get container status \"67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f\": rpc error: code = NotFound desc = could not find container \"67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f\": container with ID starting with 67ced4df76afd88582ab89a81a26135295363e1f1a5506eb943cd28d69e1f67f not found: ID does not exist"
Nov 02 22:34:30.557121 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.557095    5067 scope.go:122] "RemoveContainer" containerID="aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6"
Nov 02 22:34:30.557312 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:34:30.557293    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6\": container with ID starting with aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6 not found: ID does not exist" containerID="aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6"
Nov 02 22:34:30.557386 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.557322    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6"} err="failed to get container status \"aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6\": rpc error: code = NotFound desc = could not find container \"aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6\": container with ID starting with aa609169e6e436e581b7067ed7adf638ca19e94ce3f39b6fe0a9b7c9c0367dd6 not found: ID does not exist"
Nov 02 22:34:30.693283 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.693245    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"]
Nov 02 22:34:30.741967 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:30.741935    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bb1a7ce7-openshift-default-c77b6d4c7-swfwt"]
Nov 02 22:34:31.844751 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:31.844717    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" path="/var/lib/kubelet/pods/7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01/volumes"
Nov 02 22:34:37.576176 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:34:37.576142    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:34:44.947401 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:44.947337    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main" containerID="cri-o://bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c" gracePeriod=45
Nov 02 22:34:44.948135 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:44.947917    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" containerID="cri-o://f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4" gracePeriod=45
Nov 02 22:34:45.237454 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.237346    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:34:45.252406 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.252380    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:34:45.346117 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346078    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tls-certs\" (UniqueName: \"kubernetes.io/secret/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tls-certs\") pod \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") "
Nov 02 22:34:45.346335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346167    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-tmp-dir\") pod \"a8ead096-3c6c-487e-84a1-0803bafc984e\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") "
Nov 02 22:34:45.346335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346204    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-model-cache\") pod \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") "
Nov 02 22:34:45.346335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346237    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-home\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-home\") pod \"a8ead096-3c6c-487e-84a1-0803bafc984e\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") "
Nov 02 22:34:45.346335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346277    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-model-cache\") pod \"a8ead096-3c6c-487e-84a1-0803bafc984e\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") "
Nov 02 22:34:45.346335 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346333    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f0b9e55-929d-467a-b407-00ecfda6f7ab-kube-api-access-x2ctz\" (UniqueName: \"kubernetes.io/projected/7f0b9e55-929d-467a-b407-00ecfda6f7ab-kube-api-access-x2ctz\") pod \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") "
Nov 02 22:34:45.346643 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346414    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8ead096-3c6c-487e-84a1-0803bafc984e-tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ead096-3c6c-487e-84a1-0803bafc984e-tls-certs\") pod \"a8ead096-3c6c-487e-84a1-0803bafc984e\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") "
Nov 02 22:34:45.346643 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346467    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8ead096-3c6c-487e-84a1-0803bafc984e-kube-api-access-crgvc\" (UniqueName: \"kubernetes.io/projected/a8ead096-3c6c-487e-84a1-0803bafc984e-kube-api-access-crgvc\") pod \"a8ead096-3c6c-487e-84a1-0803bafc984e\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") "
Nov 02 22:34:45.346643 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346494    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tmp-dir\") pod \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") "
Nov 02 22:34:45.346643 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346512    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-model-cache" pod "7f0b9e55-929d-467a-b407-00ecfda6f7ab" (UID: "7f0b9e55-929d-467a-b407-00ecfda6f7ab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:45.346643 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346582    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-dshm\") pod \"a8ead096-3c6c-487e-84a1-0803bafc984e\" (UID: \"a8ead096-3c6c-487e-84a1-0803bafc984e\") "
Nov 02 22:34:45.346990 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346650    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-dshm\") pod \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") "
Nov 02 22:34:45.346990 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346721    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-home\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-home\") pod \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\" (UID: \"7f0b9e55-929d-467a-b407-00ecfda6f7ab\") "
Nov 02 22:34:45.346990 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346758    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tmp-dir" pod "7f0b9e55-929d-467a-b407-00ecfda6f7ab" (UID: "7f0b9e55-929d-467a-b407-00ecfda6f7ab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:45.346990 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346894    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-model-cache" pod "a8ead096-3c6c-487e-84a1-0803bafc984e" (UID: "a8ead096-3c6c-487e-84a1-0803bafc984e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:45.351321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.347311    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-home" pod "a8ead096-3c6c-487e-84a1-0803bafc984e" (UID: "a8ead096-3c6c-487e-84a1-0803bafc984e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:45.351321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.347847    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.351321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.347870    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.351321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.347886    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.351321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.347900    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.351321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.350097    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-home" pod "7f0b9e55-929d-467a-b407-00ecfda6f7ab" (UID: "7f0b9e55-929d-467a-b407-00ecfda6f7ab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:45.351321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.346518    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-tmp-dir" pod "a8ead096-3c6c-487e-84a1-0803bafc984e" (UID: "a8ead096-3c6c-487e-84a1-0803bafc984e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:45.351321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.350144    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8ead096-3c6c-487e-84a1-0803bafc984e-tls-certs" pod "a8ead096-3c6c-487e-84a1-0803bafc984e" (UID: "a8ead096-3c6c-487e-84a1-0803bafc984e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:34:45.351321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.350228    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-dshm" pod "a8ead096-3c6c-487e-84a1-0803bafc984e" (UID: "a8ead096-3c6c-487e-84a1-0803bafc984e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:45.351321 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.351046    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8ead096-3c6c-487e-84a1-0803bafc984e-kube-api-access-crgvc" pod "a8ead096-3c6c-487e-84a1-0803bafc984e" (UID: "a8ead096-3c6c-487e-84a1-0803bafc984e"). InnerVolumeSpecName "kube-api-access-crgvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:34:45.358808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.351467    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tls-certs" pod "7f0b9e55-929d-467a-b407-00ecfda6f7ab" (UID: "7f0b9e55-929d-467a-b407-00ecfda6f7ab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:34:45.358808 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.354082    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f0b9e55-929d-467a-b407-00ecfda6f7ab-kube-api-access-x2ctz" pod "7f0b9e55-929d-467a-b407-00ecfda6f7ab" (UID: "7f0b9e55-929d-467a-b407-00ecfda6f7ab"). InnerVolumeSpecName "kube-api-access-x2ctz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:34:45.360442 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.359248    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-dshm" pod "7f0b9e55-929d-467a-b407-00ecfda6f7ab" (UID: "7f0b9e55-929d-467a-b407-00ecfda6f7ab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:34:45.448908 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.448871    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ead096-3c6c-487e-84a1-0803bafc984e-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.448908 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.448907    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crgvc\" (UniqueName: \"kubernetes.io/projected/a8ead096-3c6c-487e-84a1-0803bafc984e-kube-api-access-crgvc\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.449187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.448920    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.449187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.448934    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.449187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.448944    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f0b9e55-929d-467a-b407-00ecfda6f7ab-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.449187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.448956    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f0b9e55-929d-467a-b407-00ecfda6f7ab-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.449187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.448970    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ead096-3c6c-487e-84a1-0803bafc984e-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.449187 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.448981    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2ctz\" (UniqueName: \"kubernetes.io/projected/7f0b9e55-929d-467a-b407-00ecfda6f7ab-kube-api-access-x2ctz\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:45.620441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.620390    5067 generic.go:356] "Generic (PLEG): container finished" podID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerID="f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4" exitCode=0
Nov 02 22:34:45.620441 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.620446    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"
Nov 02 22:34:45.620710 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.620485    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" event={"ID":"a8ead096-3c6c-487e-84a1-0803bafc984e","Type":"ContainerDied","Data":"f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4"}
Nov 02 22:34:45.620710 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.620527    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" event={"ID":"a8ead096-3c6c-487e-84a1-0803bafc984e","Type":"ContainerDied","Data":"c83027aff65bb335ad4077a51a9203596f75797d13aeb02345c7aba8ab748c8f"}
Nov 02 22:34:45.620710 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.620545    5067 scope.go:122] "RemoveContainer" containerID="f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4"
Nov 02 22:34:45.621968 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.621941    5067 generic.go:356] "Generic (PLEG): container finished" podID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerID="bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c" exitCode=0
Nov 02 22:34:45.622103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.621973    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"
Nov 02 22:34:45.622103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.622022    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" event={"ID":"7f0b9e55-929d-467a-b407-00ecfda6f7ab","Type":"ContainerDied","Data":"bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c"}
Nov 02 22:34:45.622103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.622069    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw" event={"ID":"7f0b9e55-929d-467a-b407-00ecfda6f7ab","Type":"ContainerDied","Data":"4d22c6b7a3dcc61617d317813beb47455a5897010b3a8867365491c78b40600e"}
Nov 02 22:34:45.631023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.631003    5067 scope.go:122] "RemoveContainer" containerID="f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4"
Nov 02 22:34:45.631395 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:34:45.631348    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4\": container with ID starting with f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4 not found: ID does not exist" containerID="f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4"
Nov 02 22:34:45.631513 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.631394    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4"} err="failed to get container status \"f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4\": rpc error: code = NotFound desc = could not find container \"f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4\": container with ID starting with f3192c02b0f389e04f03fe5819ab0e1a75ac3c86c9b5785f1629b38b6a321ec4 not found: ID does not exist"
Nov 02 22:34:45.631513 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.631414    5067 scope.go:122] "RemoveContainer" containerID="bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c"
Nov 02 22:34:45.642712 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.642692    5067 scope.go:122] "RemoveContainer" containerID="bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c"
Nov 02 22:34:45.642998 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:34:45.642975    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c\": container with ID starting with bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c not found: ID does not exist" containerID="bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c"
Nov 02 22:34:45.643047 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.643009    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c"} err="failed to get container status \"bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c\": rpc error: code = NotFound desc = could not find container \"bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c\": container with ID starting with bc892d416aaa127d3c2b148347621fbd9287580a1cda7826440b39784fcaaf7c not found: ID does not exist"
Nov 02 22:34:45.776518 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.776478    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"]
Nov 02 22:34:45.779863 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.779836    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/distinct-2-w2-8263a4-kserve-8c5795b84-dsfxw"]
Nov 02 22:34:45.846857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.846818    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" path="/var/lib/kubelet/pods/7f0b9e55-929d-467a-b407-00ecfda6f7ab/volumes"
Nov 02 22:34:45.884354 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.884264    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"]
Nov 02 22:34:45.886950 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:45.886923    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm"]
Nov 02 22:34:46.230993 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:46.230942    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8263a4/simulator-w2-8263a4-kserve-68cc7c8765-6k6rm" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:34:47.845053 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:47.845011    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" path="/var/lib/kubelet/pods/a8ead096-3c6c-487e-84a1-0803bafc984e/volumes"
Nov 02 22:34:57.367642 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.367603    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"]
Nov 02 22:34:57.368265 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.367832    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl" podUID="92b04e93-2dfc-4b66-868e-a4cebe5722e5" containerName="maas-api" containerID="cri-o://edb77b9e14a2a4883eb047a640ffdc502bab189cfea833bd7f20a404104a8c44" gracePeriod=30
Nov 02 22:34:57.694722 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.694534    5067 generic.go:356] "Generic (PLEG): container finished" podID="92b04e93-2dfc-4b66-868e-a4cebe5722e5" containerID="edb77b9e14a2a4883eb047a640ffdc502bab189cfea833bd7f20a404104a8c44" exitCode=0
Nov 02 22:34:57.694722 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.694603    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl" event={"ID":"92b04e93-2dfc-4b66-868e-a4cebe5722e5","Type":"ContainerDied","Data":"edb77b9e14a2a4883eb047a640ffdc502bab189cfea833bd7f20a404104a8c44"}
Nov 02 22:34:57.740786 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.740741    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:34:57.819387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.805702    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-maas-api-tls\") pod \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") "
Nov 02 22:34:57.819387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.805797    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92b04e93-2dfc-4b66-868e-a4cebe5722e5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92b04e93-2dfc-4b66-868e-a4cebe5722e5-openshift-service-ca\") pod \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") "
Nov 02 22:34:57.819387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.805894    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-metrics-certs\") pod \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") "
Nov 02 22:34:57.819387 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.805991    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92b04e93-2dfc-4b66-868e-a4cebe5722e5-kube-api-access-lz4kc\" (UniqueName: \"kubernetes.io/projected/92b04e93-2dfc-4b66-868e-a4cebe5722e5-kube-api-access-lz4kc\") pod \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\" (UID: \"92b04e93-2dfc-4b66-868e-a4cebe5722e5\") "
Nov 02 22:34:57.841433 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.831995    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92b04e93-2dfc-4b66-868e-a4cebe5722e5-openshift-service-ca" pod "92b04e93-2dfc-4b66-868e-a4cebe5722e5" (UID: "92b04e93-2dfc-4b66-868e-a4cebe5722e5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:34:57.844069 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.841920    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-maas-api-tls" pod "92b04e93-2dfc-4b66-868e-a4cebe5722e5" (UID: "92b04e93-2dfc-4b66-868e-a4cebe5722e5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:34:57.876056 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.875915    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92b04e93-2dfc-4b66-868e-a4cebe5722e5-kube-api-access-lz4kc" pod "92b04e93-2dfc-4b66-868e-a4cebe5722e5" (UID: "92b04e93-2dfc-4b66-868e-a4cebe5722e5"). InnerVolumeSpecName "kube-api-access-lz4kc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:34:57.876056 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.876002    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-metrics-certs" pod "92b04e93-2dfc-4b66-868e-a4cebe5722e5" (UID: "92b04e93-2dfc-4b66-868e-a4cebe5722e5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:34:57.910307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.910209    5067 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-metrics-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:57.910307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.910245    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz4kc\" (UniqueName: \"kubernetes.io/projected/92b04e93-2dfc-4b66-868e-a4cebe5722e5-kube-api-access-lz4kc\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:57.910307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.910263    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92b04e93-2dfc-4b66-868e-a4cebe5722e5-maas-api-tls\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:57.910307 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:57.910277    5067 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92b04e93-2dfc-4b66-868e-a4cebe5722e5-openshift-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:34:58.705251 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:58.705136    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl" event={"ID":"92b04e93-2dfc-4b66-868e-a4cebe5722e5","Type":"ContainerDied","Data":"5df3452295be86144cfb921daf7141fbf8fb2019e6dce21801d44c9fa0718be7"}
Nov 02 22:34:58.705794 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:58.705256    5067 scope.go:122] "RemoveContainer" containerID="edb77b9e14a2a4883eb047a640ffdc502bab189cfea833bd7f20a404104a8c44"
Nov 02 22:34:58.705794 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:58.705462    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"
Nov 02 22:34:58.862883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:58.862839    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"]
Nov 02 22:34:58.906730 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:58.906695    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-8263a4-5679777899-l5hbl"]
Nov 02 22:34:59.844749 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:34:59.844717    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92b04e93-2dfc-4b66-868e-a4cebe5722e5" path="/var/lib/kubelet/pods/92b04e93-2dfc-4b66-868e-a4cebe5722e5/volumes"
Nov 02 22:35:07.532012 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:07.531973    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"]
Nov 02 22:35:07.532503 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:07.532264    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerName="istio-proxy" containerID="cri-o://eb97790949981f471461ef800b00c44b968fbc89fd344b4f3ceb6010ddfa9593" gracePeriod=30
Nov 02 22:35:07.555930 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:07.555896    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"]
Nov 02 22:35:07.556167 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:07.556126    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerName="istio-proxy" containerID="cri-o://7848cc03291717a09832e994545f1d726d4a576cfdf85caa3c64b9b3ab00e756" gracePeriod=30
Nov 02 22:35:07.573630 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:07.573595    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"]
Nov 02 22:35:07.573862 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:07.573834    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy" containerID="cri-o://273903ee1db696727e323e8d4162540e95c4a5b1fd598145124ed6e3e1bc3d68" gracePeriod=30
Nov 02 22:35:07.577036 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:35:07.576994    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:35:12.784904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.784871    5067 generic.go:356] "Generic (PLEG): container finished" podID="966ac637-1148-482f-9bb6-d877b0aed346" containerID="273903ee1db696727e323e8d4162540e95c4a5b1fd598145124ed6e3e1bc3d68" exitCode=0
Nov 02 22:35:12.785333 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.784951    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" event={"ID":"966ac637-1148-482f-9bb6-d877b0aed346","Type":"ContainerDied","Data":"273903ee1db696727e323e8d4162540e95c4a5b1fd598145124ed6e3e1bc3d68"}
Nov 02 22:35:12.786823 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.786796    5067 generic.go:356] "Generic (PLEG): container finished" podID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerID="eb97790949981f471461ef800b00c44b968fbc89fd344b4f3ceb6010ddfa9593" exitCode=0
Nov 02 22:35:12.786958 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.786861    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" event={"ID":"3600f0db-a16b-4af9-beaf-3c357cf73e9d","Type":"ContainerDied","Data":"eb97790949981f471461ef800b00c44b968fbc89fd344b4f3ceb6010ddfa9593"}
Nov 02 22:35:12.788631 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.788605    5067 generic.go:356] "Generic (PLEG): container finished" podID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerID="7848cc03291717a09832e994545f1d726d4a576cfdf85caa3c64b9b3ab00e756" exitCode=0
Nov 02 22:35:12.788721 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.788651    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" event={"ID":"20ba9c3b-29f1-488e-bdbe-536c7e375bcf","Type":"ContainerDied","Data":"7848cc03291717a09832e994545f1d726d4a576cfdf85caa3c64b9b3ab00e756"}
Nov 02 22:35:12.848047 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.848013    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:35:12.876935 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.876913    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:35:12.891899 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.891877    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:35:12.944019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.943984    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-certs\") pod \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") "
Nov 02 22:35:12.944019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944020    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-certs\") pod \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") "
Nov 02 22:35:12.944248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944048    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-kube-api-access-vl4tg\" (UniqueName: \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-kube-api-access-vl4tg\") pod \"966ac637-1148-482f-9bb6-d877b0aed346\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") "
Nov 02 22:35:12.944248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944073    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-kube-api-access-vpx2d\" (UniqueName: \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-kube-api-access-vpx2d\") pod \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") "
Nov 02 22:35:12.944248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944107    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-socket\") pod \"966ac637-1148-482f-9bb6-d877b0aed346\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") "
Nov 02 22:35:12.944248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944144    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-data\") pod \"966ac637-1148-482f-9bb6-d877b0aed346\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") "
Nov 02 22:35:12.944248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944173    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-istio-token\" (UniqueName: \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-istio-token\") pod \"966ac637-1148-482f-9bb6-d877b0aed346\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") "
Nov 02 22:35:12.944248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944199    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-data\") pod \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") "
Nov 02 22:35:12.944248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944227    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-podinfo\") pod \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") "
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944253    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-credential-socket\") pod \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") "
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944293    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/966ac637-1148-482f-9bb6-d877b0aed346-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/966ac637-1148-482f-9bb6-d877b0aed346-istio-podinfo\") pod \"966ac637-1148-482f-9bb6-d877b0aed346\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") "
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944343    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-token\" (UniqueName: \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-token\") pod \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") "
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944348    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-certs" pod "3600f0db-a16b-4af9-beaf-3c357cf73e9d" (UID: "3600f0db-a16b-4af9-beaf-3c357cf73e9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944344    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-certs" pod "20ba9c3b-29f1-488e-bdbe-536c7e375bcf" (UID: "20ba9c3b-29f1-488e-bdbe-536c7e375bcf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944401    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-data\") pod \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") "
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944458    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-credential-socket\") pod \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") "
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944486    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-socket" pod "966ac637-1148-482f-9bb6-d877b0aed346" (UID: "966ac637-1148-482f-9bb6-d877b0aed346"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944504    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-podinfo\") pod \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") "
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944599    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-envoy\") pod \"966ac637-1148-482f-9bb6-d877b0aed346\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") "
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944622    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-data" pod "3600f0db-a16b-4af9-beaf-3c357cf73e9d" (UID: "3600f0db-a16b-4af9-beaf-3c357cf73e9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.944679 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944639    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-credential-socket" pod "20ba9c3b-29f1-488e-bdbe-536c7e375bcf" (UID: "20ba9c3b-29f1-488e-bdbe-536c7e375bcf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944706    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/966ac637-1148-482f-9bb6-d877b0aed346-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/966ac637-1148-482f-9bb6-d877b0aed346-istiod-ca-cert\") pod \"966ac637-1148-482f-9bb6-d877b0aed346\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") "
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944714    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-data" pod "966ac637-1148-482f-9bb6-d877b0aed346" (UID: "966ac637-1148-482f-9bb6-d877b0aed346"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944740    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-certs\") pod \"966ac637-1148-482f-9bb6-d877b0aed346\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") "
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944773    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-envoy\") pod \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") "
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944808    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-envoy\") pod \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") "
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944837    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-socket\") pod \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") "
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944872    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istiod-ca-cert\") pod \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") "
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944926    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istiod-ca-cert\") pod \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\" (UID: \"3600f0db-a16b-4af9-beaf-3c357cf73e9d\") "
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944967    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-credential-socket\") pod \"966ac637-1148-482f-9bb6-d877b0aed346\" (UID: \"966ac637-1148-482f-9bb6-d877b0aed346\") "
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.944995    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-socket\") pod \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") "
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.945031    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-kube-api-access-zwckr\" (UniqueName: \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-kube-api-access-zwckr\") pod \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") "
Nov 02 22:35:12.945287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.945078    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-token\" (UniqueName: \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-token\") pod \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\" (UID: \"20ba9c3b-29f1-488e-bdbe-536c7e375bcf\") "
Nov 02 22:35:12.945936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.945512    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:12.945936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.945533    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:12.945936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.945548    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:12.945936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.945562    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:12.945936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.945575    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:12.945936 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.945591    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:12.946860 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.946291    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-credential-socket" pod "3600f0db-a16b-4af9-beaf-3c357cf73e9d" (UID: "3600f0db-a16b-4af9-beaf-3c357cf73e9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.946860 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.946300    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-socket" pod "3600f0db-a16b-4af9-beaf-3c357cf73e9d" (UID: "3600f0db-a16b-4af9-beaf-3c357cf73e9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.946860 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.946545    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-data" pod "20ba9c3b-29f1-488e-bdbe-536c7e375bcf" (UID: "20ba9c3b-29f1-488e-bdbe-536c7e375bcf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.946860 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.946554    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-certs" pod "966ac637-1148-482f-9bb6-d877b0aed346" (UID: "966ac637-1148-482f-9bb6-d877b0aed346"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.946860 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.946677    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-socket" pod "20ba9c3b-29f1-488e-bdbe-536c7e375bcf" (UID: "20ba9c3b-29f1-488e-bdbe-536c7e375bcf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.946860 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.946783    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/966ac637-1148-482f-9bb6-d877b0aed346-istiod-ca-cert" pod "966ac637-1148-482f-9bb6-d877b0aed346" (UID: "966ac637-1148-482f-9bb6-d877b0aed346"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:35:12.947328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.946889    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-credential-socket" pod "966ac637-1148-482f-9bb6-d877b0aed346" (UID: "966ac637-1148-482f-9bb6-d877b0aed346"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.947328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.946985    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istiod-ca-cert" pod "20ba9c3b-29f1-488e-bdbe-536c7e375bcf" (UID: "20ba9c3b-29f1-488e-bdbe-536c7e375bcf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:35:12.947328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.947201    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istiod-ca-cert" pod "3600f0db-a16b-4af9-beaf-3c357cf73e9d" (UID: "3600f0db-a16b-4af9-beaf-3c357cf73e9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:35:12.947328 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.947248    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-kube-api-access-vpx2d" pod "3600f0db-a16b-4af9-beaf-3c357cf73e9d" (UID: "3600f0db-a16b-4af9-beaf-3c357cf73e9d"). InnerVolumeSpecName "kube-api-access-vpx2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:35:12.947575 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.947475    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-podinfo" pod "20ba9c3b-29f1-488e-bdbe-536c7e375bcf" (UID: "20ba9c3b-29f1-488e-bdbe-536c7e375bcf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:35:12.948109 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.948084    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-istio-token" pod "966ac637-1148-482f-9bb6-d877b0aed346" (UID: "966ac637-1148-482f-9bb6-d877b0aed346"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:35:12.948262 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.948239    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-token" pod "3600f0db-a16b-4af9-beaf-3c357cf73e9d" (UID: "3600f0db-a16b-4af9-beaf-3c357cf73e9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:35:12.948403 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.948349    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-kube-api-access-vl4tg" pod "966ac637-1148-482f-9bb6-d877b0aed346" (UID: "966ac637-1148-482f-9bb6-d877b0aed346"). InnerVolumeSpecName "kube-api-access-vl4tg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:35:12.948403 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.948390    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-podinfo" pod "3600f0db-a16b-4af9-beaf-3c357cf73e9d" (UID: "3600f0db-a16b-4af9-beaf-3c357cf73e9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:35:12.948942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.948919    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-token" pod "20ba9c3b-29f1-488e-bdbe-536c7e375bcf" (UID: "20ba9c3b-29f1-488e-bdbe-536c7e375bcf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:35:12.949205 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.949182    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/966ac637-1148-482f-9bb6-d877b0aed346-istio-podinfo" pod "966ac637-1148-482f-9bb6-d877b0aed346" (UID: "966ac637-1148-482f-9bb6-d877b0aed346"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 22:35:12.949293 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.949199    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-kube-api-access-zwckr" pod "20ba9c3b-29f1-488e-bdbe-536c7e375bcf" (UID: "20ba9c3b-29f1-488e-bdbe-536c7e375bcf"). InnerVolumeSpecName "kube-api-access-zwckr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:35:12.949293 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.949223    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-envoy" pod "966ac637-1148-482f-9bb6-d877b0aed346" (UID: "966ac637-1148-482f-9bb6-d877b0aed346"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.949608 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.949340    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-envoy" pod "20ba9c3b-29f1-488e-bdbe-536c7e375bcf" (UID: "20ba9c3b-29f1-488e-bdbe-536c7e375bcf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:12.949769 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:12.949748    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-envoy" pod "3600f0db-a16b-4af9-beaf-3c357cf73e9d" (UID: "3600f0db-a16b-4af9-beaf-3c357cf73e9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:35:13.046212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046177    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/966ac637-1148-482f-9bb6-d877b0aed346-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046206    5067 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-workload-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046216    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046225    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046234    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046245    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046254    5067 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istiod-ca-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046263    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046271    5067 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-workload-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046280    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwckr\" (UniqueName: \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-kube-api-access-zwckr\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046289    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046297    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl4tg\" (UniqueName: \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-kube-api-access-vl4tg\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046306    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpx2d\" (UniqueName: \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-kube-api-access-vpx2d\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046338    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/966ac637-1148-482f-9bb6-d877b0aed346-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046347    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046355    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/966ac637-1148-482f-9bb6-d877b0aed346-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046375    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046383    5067 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20ba9c3b-29f1-488e-bdbe-536c7e375bcf-istio-data\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046392    5067 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3600f0db-a16b-4af9-beaf-3c357cf73e9d-credential-socket\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046401    5067 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3600f0db-a16b-4af9-beaf-3c357cf73e9d-istio-podinfo\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.046467 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.046408    5067 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/966ac637-1148-482f-9bb6-d877b0aed346-istio-envoy\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:35:13.795039 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.795008    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"
Nov 02 22:35:13.795509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.795007    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj" event={"ID":"966ac637-1148-482f-9bb6-d877b0aed346","Type":"ContainerDied","Data":"9319e4c75ab4254586d3b67dc217cb418570897086b1689cec05e3a727901503"}
Nov 02 22:35:13.795509 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.795152    5067 scope.go:122] "RemoveContainer" containerID="273903ee1db696727e323e8d4162540e95c4a5b1fd598145124ed6e3e1bc3d68"
Nov 02 22:35:13.796573 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.796527    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"
Nov 02 22:35:13.796573 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.796551    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j" event={"ID":"3600f0db-a16b-4af9-beaf-3c357cf73e9d","Type":"ContainerDied","Data":"57d110290bcc58797cbd5199bf17460c38952bb55cd0e316f8a64a36126b9a16"}
Nov 02 22:35:13.798290 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.798264    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p" event={"ID":"20ba9c3b-29f1-488e-bdbe-536c7e375bcf","Type":"ContainerDied","Data":"59f3f6e22e2ba8c2f29f2a6ffd51ffa89a753dd18abb212ba629acbecaab7646"}
Nov 02 22:35:13.798459 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.798278    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"
Nov 02 22:35:13.806048 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.806024    5067 scope.go:122] "RemoveContainer" containerID="eb97790949981f471461ef800b00c44b968fbc89fd344b4f3ceb6010ddfa9593"
Nov 02 22:35:13.814813 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.814796    5067 scope.go:122] "RemoveContainer" containerID="7848cc03291717a09832e994545f1d726d4a576cfdf85caa3c64b9b3ab00e756"
Nov 02 22:35:13.952666 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.952631    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"]
Nov 02 22:35:13.999040 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:13.999005    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-r9m7j"]
Nov 02 22:35:14.101925 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:14.101839    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"]
Nov 02 22:35:14.107286 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:14.107259    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-rdv6p"]
Nov 02 22:35:14.210591 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:14.210556    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"]
Nov 02 22:35:14.216096 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:14.216067    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-8263a4-openshift-default-5bf6d64689-jm7zj"]
Nov 02 22:35:15.847720 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:15.847675    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" path="/var/lib/kubelet/pods/20ba9c3b-29f1-488e-bdbe-536c7e375bcf/volumes"
Nov 02 22:35:15.848475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:15.848452    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" path="/var/lib/kubelet/pods/3600f0db-a16b-4af9-beaf-3c357cf73e9d/volumes"
Nov 02 22:35:15.849140 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:15.849119    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="966ac637-1148-482f-9bb6-d877b0aed346" path="/var/lib/kubelet/pods/966ac637-1148-482f-9bb6-d877b0aed346/volumes"
Nov 02 22:35:27.263601 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:27.263541    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:35:27.264552 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:27.264500    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:35:28.438858 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:28.438811    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:35:28.455889 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:35:28.455863    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:35:37.576129 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:35:37.576085    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:36:07.578326 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:36:07.578284    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:36:37.575297 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:36:37.575257    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:36:41.845221 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:41.845191    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"]
Nov 02 22:36:41.845816 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:41.845393    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v" podUID="1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" containerName="discovery" containerID="cri-o://e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721" gracePeriod=30
Nov 02 22:36:42.115288 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.115262    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:36:42.221306 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.221267    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-ca-configmap\") pod \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") "
Nov 02 22:36:42.221520 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.221347    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-dns-cert\") pod \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") "
Nov 02 22:36:42.221520 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.221395    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-token\" (UniqueName: \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-token\") pod \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") "
Nov 02 22:36:42.221520 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.221429    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-local-certs\") pod \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") "
Nov 02 22:36:42.221520 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.221452    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-kubeconfig\") pod \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") "
Nov 02 22:36:42.221520 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.221479    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-cacerts\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-cacerts\") pod \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") "
Nov 02 22:36:42.221520 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.221514    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-kube-api-access-qv8dp\" (UniqueName: \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-kube-api-access-qv8dp\") pod \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\" (UID: \"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907\") "
Nov 02 22:36:42.221874 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.221786    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-ca-configmap" pod "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" (UID: "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:36:42.221997 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.221963    5067 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-ca-configmap\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:36:42.223880 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.223852    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-dns-cert" pod "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" (UID: "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:36:42.224019 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.223914    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-cacerts" pod "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" (UID: "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:36:42.224105 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.224078    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-local-certs" pod "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" (UID: "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:36:42.224248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.224217    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-kube-api-access-qv8dp" pod "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" (UID: "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907"). InnerVolumeSpecName "kube-api-access-qv8dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:36:42.224327 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.224253    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-kubeconfig" pod "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" (UID: "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:36:42.224327 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.224264    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-token" pod "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" (UID: "1e8004d9-e8cc-4e69-b78b-8c9d74c2e907"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:36:42.269990 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.269954    5067 generic.go:356] "Generic (PLEG): container finished" podID="1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" containerID="e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721" exitCode=0
Nov 02 22:36:42.270146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.270004    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"
Nov 02 22:36:42.270146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.270047    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v" event={"ID":"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907","Type":"ContainerDied","Data":"e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721"}
Nov 02 22:36:42.270146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.270110    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v" event={"ID":"1e8004d9-e8cc-4e69-b78b-8c9d74c2e907","Type":"ContainerDied","Data":"723a0a5d538392890306bc1eb48e3d2cd8871db4c6aae0364f7c0c70d93af43e"}
Nov 02 22:36:42.270146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.270142    5067 scope.go:122] "RemoveContainer" containerID="e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721"
Nov 02 22:36:42.280243 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.280225    5067 scope.go:122] "RemoveContainer" containerID="e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721"
Nov 02 22:36:42.280553 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:36:42.280536    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721\": container with ID starting with e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721 not found: ID does not exist" containerID="e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721"
Nov 02 22:36:42.280622 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.280562    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721"} err="failed to get container status \"e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721\": rpc error: code = NotFound desc = could not find container \"e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721\": container with ID starting with e470f5bec8cbd9546ff5f54617bb72270d65932e46d81b0dd38b7b0e05dc3721 not found: ID does not exist"
Nov 02 22:36:42.323439 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.323409    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv8dp\" (UniqueName: \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-kube-api-access-qv8dp\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:36:42.323439 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.323435    5067 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-csr-dns-cert\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:36:42.323620 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.323447    5067 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-token\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:36:42.323620 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.323456    5067 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-local-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:36:42.323620 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.323464    5067 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-istio-kubeconfig\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:36:42.323620 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.323472    5067 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907-cacerts\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:36:42.424977 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.424882    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"]
Nov 02 22:36:42.431089 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:42.431057    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-np74v"]
Nov 02 22:36:43.844157 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:36:43.844113    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" path="/var/lib/kubelet/pods/1e8004d9-e8cc-4e69-b78b-8c9d74c2e907/volumes"
Nov 02 22:37:07.589698 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:37:07.589655    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:37:37.577569 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:37:37.577521    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:38:04.050034 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:38:04.049994    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"]
Nov 02 22:38:04.096883 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:38:04.096838    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"]
Nov 02 22:38:04.098747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:38:04.098712    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-74121fcd-2c9sn"]
Nov 02 22:38:04.100218 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:38:04.100194    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4f91c03-qg2wb"]
Nov 02 22:38:05.848329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:38:05.848287    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5feca907-056a-416b-8cf7-246fc20b2194" path="/var/lib/kubelet/pods/5feca907-056a-416b-8cf7-246fc20b2194/volumes"
Nov 02 22:38:05.848935 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:38:05.848910    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b0584fc-060b-4c59-a5ff-34a7ae51b1d0" path="/var/lib/kubelet/pods/9b0584fc-060b-4c59-a5ff-34a7ae51b1d0/volumes"
Nov 02 22:38:07.718417 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:38:07.718359    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:38:37.731537 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:38:37.731486    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:39:07.576127 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:39:07.576085    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:39:28.851955 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:39:28.851922    5067 scope.go:122] "RemoveContainer" containerID="754d22b8d78d2fa14ee33207ea12addcd98b49b0d42ce5d0dcc34798b4c44388"
Nov 02 22:39:28.863960 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:39:28.863938    5067 scope.go:122] "RemoveContainer" containerID="077c22cd5c3fee756c13bd53c58b8f40b8f1e8642c104293f4090fddd5dd173c"
Nov 02 22:39:37.576144 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:39:37.576105    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:40:07.575574 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:40:07.575529    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:40:28.601394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:40:28.599835    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:40:28.601394 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:40:28.600639    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:40:37.575695 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:40:37.575650    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:40:49.821397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:40:49.821060    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-fzph7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:40:49.821397 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:40:49.821146    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-fzph7" podUID="790b1bdf-449a-4671-b640-97990dd0d405" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:41:07.589315 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:41:07.589254    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:41:37.575121 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:41:37.575085    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:42:07.576469 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:42:07.576426    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:42:37.576622 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:42:37.576580    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:43:07.576551 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:43:07.576508    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:43:12.256390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:43:12.255507    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-4jg6k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:43:12.256390 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:43:12.255600    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-4jg6k" podUID="1d4d7818-f4be-42f8-9db2-28dcbf96091d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:43:37.575414 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:43:37.575349    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:44:02.288518 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288480    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"]
Nov 02 22:44:02.288942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288880    5067 cpu_manager.go:414] "containerMap: removing container" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy"
Nov 02 22:44:02.288942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288892    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy"
Nov 02 22:44:02.288942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288910    5067 cpu_manager.go:414] "containerMap: removing container" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerName="istio-proxy"
Nov 02 22:44:02.288942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288916    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerName="istio-proxy"
Nov 02 22:44:02.288942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288928    5067 cpu_manager.go:414] "containerMap: removing container" podUID="f3880123-ed80-4cec-8369-998133747622" containerName="maas-api"
Nov 02 22:44:02.288942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288932    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3880123-ed80-4cec-8369-998133747622" containerName="maas-api"
Nov 02 22:44:02.288942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288939    5067 cpu_manager.go:414] "containerMap: removing container" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy"
Nov 02 22:44:02.288942 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288944    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288952    5067 cpu_manager.go:414] "containerMap: removing container" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288957    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288964    5067 cpu_manager.go:414] "containerMap: removing container" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288969    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288974    5067 cpu_manager.go:414] "containerMap: removing container" podUID="9b0584fc-060b-4c59-a5ff-34a7ae51b1d0" containerName="revoke-keys"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288980    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b0584fc-060b-4c59-a5ff-34a7ae51b1d0" containerName="revoke-keys"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288986    5067 cpu_manager.go:414] "containerMap: removing container" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288991    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.288998    5067 cpu_manager.go:414] "containerMap: removing container" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289002    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289008    5067 cpu_manager.go:414] "containerMap: removing container" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289019    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289030    5067 cpu_manager.go:414] "containerMap: removing container" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289038    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289044    5067 cpu_manager.go:414] "containerMap: removing container" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289049    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289057    5067 cpu_manager.go:414] "containerMap: removing container" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289062    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289068    5067 cpu_manager.go:414] "containerMap: removing container" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289072    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289078    5067 cpu_manager.go:414] "containerMap: removing container" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289084    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289091    5067 cpu_manager.go:414] "containerMap: removing container" podUID="5feca907-056a-416b-8cf7-246fc20b2194" containerName="revoke-keys"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289095    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5feca907-056a-416b-8cf7-246fc20b2194" containerName="revoke-keys"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289104    5067 cpu_manager.go:414] "containerMap: removing container" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289111    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289120    5067 cpu_manager.go:414] "containerMap: removing container" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289124    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289130    5067 cpu_manager.go:414] "containerMap: removing container" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289134    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289139    5067 cpu_manager.go:414] "containerMap: removing container" podUID="1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" containerName="discovery"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289145    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" containerName="discovery"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289155    5067 cpu_manager.go:414] "containerMap: removing container" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289159    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289168    5067 cpu_manager.go:414] "containerMap: removing container" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289173    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289183    5067 cpu_manager.go:414] "containerMap: removing container" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289190    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289202    5067 cpu_manager.go:414] "containerMap: removing container" podUID="92b04e93-2dfc-4b66-868e-a4cebe5722e5" containerName="maas-api"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289207    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92b04e93-2dfc-4b66-868e-a4cebe5722e5" containerName="maas-api"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289213    5067 cpu_manager.go:414] "containerMap: removing container" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289217    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289226    5067 cpu_manager.go:414] "containerMap: removing container" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289230    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289239    5067 cpu_manager.go:414] "containerMap: removing container" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289243    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289308    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="af6506ab-99cf-4bc1-89b8-3340e6f0c411" containerName="main"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289317    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289324    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="06409ed1-ce32-4d21-b844-b18b32615f3d" containerName="istio-proxy"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289331    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="5feca907-056a-416b-8cf7-246fc20b2194" containerName="revoke-keys"
Nov 02 22:44:02.289308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289337    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="59ed629b-1c29-4cfc-909c-7b4089c141ac" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289343    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="20ba9c3b-29f1-488e-bdbe-536c7e375bcf" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289352    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="966ac637-1148-482f-9bb6-d877b0aed346" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289361    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e8004d9-e8cc-4e69-b78b-8c9d74c2e907" containerName="discovery"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289382    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="3600f0db-a16b-4af9-beaf-3c357cf73e9d" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289387    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="deca162c-c1be-4048-89f4-1ad60fd1ee87" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289392    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a1952c2-04f5-40eb-96a0-93a7e9ea1ff6" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289397    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289402    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="92b04e93-2dfc-4b66-868e-a4cebe5722e5" containerName="maas-api"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289407    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8ead096-3c6c-487e-84a1-0803bafc984e" containerName="main"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289411    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3a2f228-12a8-4e72-b32e-9b7fcb223f0b" containerName="main"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289418    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="40a9abc6-a069-44c5-b62a-b700c8994560" containerName="main"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289422    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="7326e5f6-1330-453b-99a0-1eae9a0c89fc" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289428    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="0068f3ef-ef4d-47d1-a79b-9fa8ea69461a" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289435    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3880123-ed80-4cec-8369-998133747622" containerName="maas-api"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289439    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d7463af-b57c-45a2-a286-b7a70fadb525" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289445    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="e68eb745-1a75-4eaf-9569-54624bd97223" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289449    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b0584fc-060b-4c59-a5ff-34a7ae51b1d0" containerName="revoke-keys"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289455    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="e71caf86-9d04-4442-bcfd-5b7bf21900e6" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289460    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="69e0edee-4e5a-4d8a-8c1b-8407f2a8ed04" containerName="istio-proxy"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289465    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f0b9e55-929d-467a-b407-00ecfda6f7ab" containerName="main"
Nov 02 22:44:02.290885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.289471    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b203dd1-35c7-42d9-993d-d515b473a2f1" containerName="istio-proxy"
Nov 02 22:44:02.292486 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.292470    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.387943 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.387908    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-a8baae\"/\"openshift-service-ca.crt\""
Nov 02 22:44:02.388113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.387970    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-a8baae\"/\"kube-root-ca.crt\""
Nov 02 22:44:02.388113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.388000    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-a8baae\"/\"default-dockercfg-gwzxg\""
Nov 02 22:44:02.388113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.388009    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-a8baae\"/\"simulator-single-main-a8baae-kserve-self-signed-certs\""
Nov 02 22:44:02.390046 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.390026    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"]
Nov 02 22:44:02.438213 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.438157    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbnxv\" (UniqueName: \"kubernetes.io/projected/1fdee942-dbbd-476f-85de-82cff84840da-kube-api-access-qbnxv\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.438213 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.438215    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fdee942-dbbd-476f-85de-82cff84840da-tls-certs\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.438591 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.438244    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-dshm\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.438591 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.438292    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-model-cache\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.438591 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.438323    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-home\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.438591 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.438405    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-tmp-dir\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.539948 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.539858    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnxv\" (UniqueName: \"kubernetes.io/projected/1fdee942-dbbd-476f-85de-82cff84840da-kube-api-access-qbnxv\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.539948 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.539914    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fdee942-dbbd-476f-85de-82cff84840da-tls-certs\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.539948 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.539944    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-dshm\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.540213 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.540064    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-model-cache\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.540213 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.540118    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-home\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.540213 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.540201    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-tmp-dir\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.540569 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.540544    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-model-cache\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.540677 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.540592    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-home\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.540677 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.540625    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-tmp-dir\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.542287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.542265    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-dshm\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.543039 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.543015    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fdee942-dbbd-476f-85de-82cff84840da-tls-certs\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.601747 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.601708    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbnxv\" (UniqueName: \"kubernetes.io/projected/1fdee942-dbbd-476f-85de-82cff84840da-kube-api-access-qbnxv\") pod \"simulator-single-main-a8baae-kserve-66697f5b48-vfnnj\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") " pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.604348 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.604325    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:02.986760 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:02.986603    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"]
Nov 02 22:44:02.989676 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:44:02.989648    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fdee942_dbbd_476f_85de_82cff84840da.slice/crio-0fad4cbf0bfe472fa2e45905cb290eb59194b85a3282fbc0885035803a2b31e2 WatchSource:0}: Error finding container 0fad4cbf0bfe472fa2e45905cb290eb59194b85a3282fbc0885035803a2b31e2: Status 404 returned error can't find the container with id 0fad4cbf0bfe472fa2e45905cb290eb59194b85a3282fbc0885035803a2b31e2
Nov 02 22:44:03.057318 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:03.057292    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj" event={"ID":"1fdee942-dbbd-476f-85de-82cff84840da","Type":"ContainerStarted","Data":"3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694"}
Nov 02 22:44:03.057478 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:03.057333    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj" event={"ID":"1fdee942-dbbd-476f-85de-82cff84840da","Type":"ContainerStarted","Data":"0fad4cbf0bfe472fa2e45905cb290eb59194b85a3282fbc0885035803a2b31e2"}
Nov 02 22:44:04.173670 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:04.173615    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj" podStartSLOduration=2.173600898 podStartE2EDuration="2.173600898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:44:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:44:04.172757953 +0000 UTC m=+2316.456567639" watchObservedRunningTime="2026-11-02 22:44:04.173600898 +0000 UTC m=+2316.457410579"
Nov 02 22:44:04.605059 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:04.605021    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:07.583037 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:44:07.582990    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:44:12.605471 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:12.605428    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:12.618167 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:12.618134    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:13.126355 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:44:13.126321    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:44:37.576320 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:44:37.576275    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:45:00.154718 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.154678    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"]
Nov 02 22:45:00.155336 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.155105    5067 cpu_manager.go:414] "containerMap: removing container" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy"
Nov 02 22:45:00.155336 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.155116    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy"
Nov 02 22:45:00.155336 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.155215    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="7da39c9c-fdc0-4899-9dd5-9ea4ab06ed01" containerName="istio-proxy"
Nov 02 22:45:00.155336 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.155225    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4bc6e5d-e119-42e0-8254-df6772091d7e" containerName="istio-proxy"
Nov 02 22:45:00.158391 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.158344    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"
Nov 02 22:45:00.207305 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.207271    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 22:45:00.221704 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.221673    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8ncx\" (UniqueName: \"kubernetes.io/projected/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-kube-api-access-p8ncx\") pod \"maas-api-key-cleanup-29894325-ll9qf\" (UID: \"43a94e39-5f1a-46ad-9ba4-1cff74c726b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"
Nov 02 22:45:00.221843 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.221709    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-maas-api-service-ca\") pod \"maas-api-key-cleanup-29894325-ll9qf\" (UID: \"43a94e39-5f1a-46ad-9ba4-1cff74c726b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"
Nov 02 22:45:00.253764 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.253730    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cndr2\""
Nov 02 22:45:00.253764 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.253736    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 22:45:00.255938 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.255912    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"]
Nov 02 22:45:00.322662 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.322628    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ncx\" (UniqueName: \"kubernetes.io/projected/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-kube-api-access-p8ncx\") pod \"maas-api-key-cleanup-29894325-ll9qf\" (UID: \"43a94e39-5f1a-46ad-9ba4-1cff74c726b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"
Nov 02 22:45:00.322662 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.322664    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-maas-api-service-ca\") pod \"maas-api-key-cleanup-29894325-ll9qf\" (UID: \"43a94e39-5f1a-46ad-9ba4-1cff74c726b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"
Nov 02 22:45:00.323287 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.323266    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-maas-api-service-ca\") pod \"maas-api-key-cleanup-29894325-ll9qf\" (UID: \"43a94e39-5f1a-46ad-9ba4-1cff74c726b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"
Nov 02 22:45:00.380625 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.380594    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8ncx\" (UniqueName: \"kubernetes.io/projected/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-kube-api-access-p8ncx\") pod \"maas-api-key-cleanup-29894325-ll9qf\" (UID: \"43a94e39-5f1a-46ad-9ba4-1cff74c726b1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"
Nov 02 22:45:00.472264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.472235    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"
Nov 02 22:45:00.838473 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:00.838351    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"]
Nov 02 22:45:00.841644 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:45:00.841606    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a94e39_5f1a_46ad_9ba4_1cff74c726b1.slice/crio-79f6a424f50467e36da8ba6dca2149014f837318eccfb3bfb8ddd069eb0a90e0 WatchSource:0}: Error finding container 79f6a424f50467e36da8ba6dca2149014f837318eccfb3bfb8ddd069eb0a90e0: Status 404 returned error can't find the container with id 79f6a424f50467e36da8ba6dca2149014f837318eccfb3bfb8ddd069eb0a90e0
Nov 02 22:45:01.354066 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:01.354025    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" event={"ID":"43a94e39-5f1a-46ad-9ba4-1cff74c726b1","Type":"ContainerStarted","Data":"bd321fc37915fec6dc8bf640e30c49f5f16eeb862d758a493cb7354768520f1f"}
Nov 02 22:45:01.354540 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:01.354088    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" event={"ID":"43a94e39-5f1a-46ad-9ba4-1cff74c726b1","Type":"ContainerStarted","Data":"79f6a424f50467e36da8ba6dca2149014f837318eccfb3bfb8ddd069eb0a90e0"}
Nov 02 22:45:01.462127 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:01.462014    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" podStartSLOduration=1.461998623 podStartE2EDuration="1.461998623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:45:01.461701305 +0000 UTC m=+2373.745510993" watchObservedRunningTime="2026-11-02 22:45:01.461998623 +0000 UTC m=+2373.745808309"
Nov 02 22:45:02.363392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:02.363343    5067 generic.go:356] "Generic (PLEG): container finished" podID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerID="bd321fc37915fec6dc8bf640e30c49f5f16eeb862d758a493cb7354768520f1f" exitCode=60
Nov 02 22:45:02.363826 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:02.363398    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" event={"ID":"43a94e39-5f1a-46ad-9ba4-1cff74c726b1","Type":"ContainerDied","Data":"bd321fc37915fec6dc8bf640e30c49f5f16eeb862d758a493cb7354768520f1f"}
Nov 02 22:45:02.363826 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:02.363674    5067 scope.go:122] "RemoveContainer" containerID="bd321fc37915fec6dc8bf640e30c49f5f16eeb862d758a493cb7354768520f1f"
Nov 02 22:45:02.970997 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:02.970965    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"]
Nov 02 22:45:03.370404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:03.370212    5067 generic.go:356] "Generic (PLEG): container finished" podID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerID="21508b41e8232f500d50746ee97ddf6bc8e777b5d71987910b6d9dfc1a190d59" exitCode=60
Nov 02 22:45:03.370404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:03.370297    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" event={"ID":"43a94e39-5f1a-46ad-9ba4-1cff74c726b1","Type":"ContainerDied","Data":"21508b41e8232f500d50746ee97ddf6bc8e777b5d71987910b6d9dfc1a190d59"}
Nov 02 22:45:03.370404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:03.370392    5067 scope.go:122] "RemoveContainer" containerID="bd321fc37915fec6dc8bf640e30c49f5f16eeb862d758a493cb7354768520f1f"
Nov 02 22:45:03.370990 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:03.370627    5067 scope.go:122] "RemoveContainer" containerID="21508b41e8232f500d50746ee97ddf6bc8e777b5d71987910b6d9dfc1a190d59"
Nov 02 22:45:03.370990 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:45:03.370871    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29894325-ll9qf_odh-ai-gateway-infra(43a94e39-5f1a-46ad-9ba4-1cff74c726b1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1"
Nov 02 22:45:04.376801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:04.376771    5067 scope.go:122] "RemoveContainer" containerID="21508b41e8232f500d50746ee97ddf6bc8e777b5d71987910b6d9dfc1a190d59"
Nov 02 22:45:04.377196 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:45:04.377013    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29894325-ll9qf_odh-ai-gateway-infra(43a94e39-5f1a-46ad-9ba4-1cff74c726b1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1"
Nov 02 22:45:07.579205 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:45:07.579156    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:45:14.839189 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:14.839151    5067 scope.go:122] "RemoveContainer" containerID="21508b41e8232f500d50746ee97ddf6bc8e777b5d71987910b6d9dfc1a190d59"
Nov 02 22:45:15.432923 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:15.432841    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" event={"ID":"43a94e39-5f1a-46ad-9ba4-1cff74c726b1","Type":"ContainerStarted","Data":"70ae8cda85c47a41d7035903bf866bbb07c48404e134833816a858dee1516b53"}
Nov 02 22:45:16.439057 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:16.439023    5067 generic.go:356] "Generic (PLEG): container finished" podID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerID="70ae8cda85c47a41d7035903bf866bbb07c48404e134833816a858dee1516b53" exitCode=60
Nov 02 22:45:16.439590 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:16.439104    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" event={"ID":"43a94e39-5f1a-46ad-9ba4-1cff74c726b1","Type":"ContainerDied","Data":"70ae8cda85c47a41d7035903bf866bbb07c48404e134833816a858dee1516b53"}
Nov 02 22:45:16.439590 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:16.439184    5067 scope.go:122] "RemoveContainer" containerID="21508b41e8232f500d50746ee97ddf6bc8e777b5d71987910b6d9dfc1a190d59"
Nov 02 22:45:16.439590 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:16.439556    5067 scope.go:122] "RemoveContainer" containerID="70ae8cda85c47a41d7035903bf866bbb07c48404e134833816a858dee1516b53"
Nov 02 22:45:16.439832 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:45:16.439814    5067 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29894325-ll9qf_odh-ai-gateway-infra(43a94e39-5f1a-46ad-9ba4-1cff74c726b1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1"
Nov 02 22:45:16.560515 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:16.560480    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"]
Nov 02 22:45:17.580069 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:17.580035    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"
Nov 02 22:45:17.691605 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:17.691566    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-kube-api-access-p8ncx\" (UniqueName: \"kubernetes.io/projected/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-kube-api-access-p8ncx\") pod \"43a94e39-5f1a-46ad-9ba4-1cff74c726b1\" (UID: \"43a94e39-5f1a-46ad-9ba4-1cff74c726b1\") "
Nov 02 22:45:17.692047 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:17.691678    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-maas-api-service-ca\") pod \"43a94e39-5f1a-46ad-9ba4-1cff74c726b1\" (UID: \"43a94e39-5f1a-46ad-9ba4-1cff74c726b1\") "
Nov 02 22:45:17.692351 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:17.692321    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-maas-api-service-ca" pod "43a94e39-5f1a-46ad-9ba4-1cff74c726b1" (UID: "43a94e39-5f1a-46ad-9ba4-1cff74c726b1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:45:17.694331 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:17.694301    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-kube-api-access-p8ncx" pod "43a94e39-5f1a-46ad-9ba4-1cff74c726b1" (UID: "43a94e39-5f1a-46ad-9ba4-1cff74c726b1"). InnerVolumeSpecName "kube-api-access-p8ncx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:45:17.792804 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:17.792769    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-maas-api-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:45:17.792804 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:17.792798    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8ncx\" (UniqueName: \"kubernetes.io/projected/43a94e39-5f1a-46ad-9ba4-1cff74c726b1-kube-api-access-p8ncx\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:45:17.991172 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:17.991123    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj" podUID="1fdee942-dbbd-476f-85de-82cff84840da" containerName="main" containerID="cri-o://3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694" gracePeriod=45
Nov 02 22:45:18.115535 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.115495    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj" podUID="1fdee942-dbbd-476f-85de-82cff84840da" containerName="main" probeResult="failure" output="Get \"https://10.132.0.106:8000/ready\": dial tcp 10.132.0.106:8000: connect: connection refused"
Nov 02 22:45:18.239479 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.239456    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:45:18.298801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.298709    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-tmp-dir\") pod \"1fdee942-dbbd-476f-85de-82cff84840da\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") "
Nov 02 22:45:18.298801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.298745    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-dshm\") pod \"1fdee942-dbbd-476f-85de-82cff84840da\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") "
Nov 02 22:45:18.298801 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.298775    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-home\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-home\") pod \"1fdee942-dbbd-476f-85de-82cff84840da\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") "
Nov 02 22:45:18.299103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.298820    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fdee942-dbbd-476f-85de-82cff84840da-kube-api-access-qbnxv\" (UniqueName: \"kubernetes.io/projected/1fdee942-dbbd-476f-85de-82cff84840da-kube-api-access-qbnxv\") pod \"1fdee942-dbbd-476f-85de-82cff84840da\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") "
Nov 02 22:45:18.299103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.298843    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-model-cache\") pod \"1fdee942-dbbd-476f-85de-82cff84840da\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") "
Nov 02 22:45:18.299103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.298908    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1fdee942-dbbd-476f-85de-82cff84840da-tls-certs\" (UniqueName: \"kubernetes.io/secret/1fdee942-dbbd-476f-85de-82cff84840da-tls-certs\") pod \"1fdee942-dbbd-476f-85de-82cff84840da\" (UID: \"1fdee942-dbbd-476f-85de-82cff84840da\") "
Nov 02 22:45:18.299103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.299037    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-tmp-dir" pod "1fdee942-dbbd-476f-85de-82cff84840da" (UID: "1fdee942-dbbd-476f-85de-82cff84840da"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:45:18.299103 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.299080    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-home" pod "1fdee942-dbbd-476f-85de-82cff84840da" (UID: "1fdee942-dbbd-476f-85de-82cff84840da"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:45:18.299360 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.299212    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-model-cache" pod "1fdee942-dbbd-476f-85de-82cff84840da" (UID: "1fdee942-dbbd-476f-85de-82cff84840da"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:45:18.299435 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.299404    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:45:18.299435 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.299422    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:45:18.299503 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.299434    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:45:18.301188 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.301164    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fdee942-dbbd-476f-85de-82cff84840da-tls-certs" pod "1fdee942-dbbd-476f-85de-82cff84840da" (UID: "1fdee942-dbbd-476f-85de-82cff84840da"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:45:18.301264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.301203    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-dshm" pod "1fdee942-dbbd-476f-85de-82cff84840da" (UID: "1fdee942-dbbd-476f-85de-82cff84840da"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:45:18.301264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.301213    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fdee942-dbbd-476f-85de-82cff84840da-kube-api-access-qbnxv" pod "1fdee942-dbbd-476f-85de-82cff84840da" (UID: "1fdee942-dbbd-476f-85de-82cff84840da"). InnerVolumeSpecName "kube-api-access-qbnxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:45:18.400329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.400290    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbnxv\" (UniqueName: \"kubernetes.io/projected/1fdee942-dbbd-476f-85de-82cff84840da-kube-api-access-qbnxv\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:45:18.400329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.400321    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fdee942-dbbd-476f-85de-82cff84840da-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:45:18.400329 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.400332    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fdee942-dbbd-476f-85de-82cff84840da-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:45:18.452882 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.452844    5067 generic.go:356] "Generic (PLEG): container finished" podID="1fdee942-dbbd-476f-85de-82cff84840da" containerID="3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694" exitCode=0
Nov 02 22:45:18.453045 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.452886    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"
Nov 02 22:45:18.453045 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.452922    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj" event={"ID":"1fdee942-dbbd-476f-85de-82cff84840da","Type":"ContainerDied","Data":"3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694"}
Nov 02 22:45:18.453045 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.452970    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj" event={"ID":"1fdee942-dbbd-476f-85de-82cff84840da","Type":"ContainerDied","Data":"0fad4cbf0bfe472fa2e45905cb290eb59194b85a3282fbc0885035803a2b31e2"}
Nov 02 22:45:18.453045 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.452990    5067 scope.go:122] "RemoveContainer" containerID="3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694"
Nov 02 22:45:18.454452 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.454393    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf" event={"ID":"43a94e39-5f1a-46ad-9ba4-1cff74c726b1","Type":"ContainerDied","Data":"79f6a424f50467e36da8ba6dca2149014f837318eccfb3bfb8ddd069eb0a90e0"}
Nov 02 22:45:18.454452 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.454408    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"
Nov 02 22:45:18.464621 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.464604    5067 scope.go:122] "RemoveContainer" containerID="3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694"
Nov 02 22:45:18.464882 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:45:18.464859    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694\": container with ID starting with 3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694 not found: ID does not exist" containerID="3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694"
Nov 02 22:45:18.464925 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.464895    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694"} err="failed to get container status \"3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694\": rpc error: code = NotFound desc = could not find container \"3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694\": container with ID starting with 3739ae170f3d30e39e3fd9729326654f125f89a1afc15b9fa5e6f2400c584694 not found: ID does not exist"
Nov 02 22:45:18.464925 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.464919    5067 scope.go:122] "RemoveContainer" containerID="70ae8cda85c47a41d7035903bf866bbb07c48404e134833816a858dee1516b53"
Nov 02 22:45:18.607491 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.607451    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"]
Nov 02 22:45:18.612527 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.612493    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-a8baae/simulator-single-main-a8baae-kserve-66697f5b48-vfnnj"]
Nov 02 22:45:18.714799 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.714763    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"]
Nov 02 22:45:18.718748 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:18.718722    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29894325-ll9qf"]
Nov 02 22:45:19.847352 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:19.847313    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1fdee942-dbbd-476f-85de-82cff84840da" path="/var/lib/kubelet/pods/1fdee942-dbbd-476f-85de-82cff84840da/volumes"
Nov 02 22:45:19.850622 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:19.847918    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" path="/var/lib/kubelet/pods/43a94e39-5f1a-46ad-9ba4-1cff74c726b1/volumes"
Nov 02 22:45:28.695279 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:28.695249    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:45:28.698030 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:28.698005    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:45:37.575565 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:45:37.575526    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:45:58.656904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:58.656865    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"]
Nov 02 22:45:58.657392 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:58.657071    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k" podUID="524851bd-08e5-4413-8e20-4e893631f23e" containerName="manager" containerID="cri-o://038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4" gracePeriod=10
Nov 02 22:45:58.914002 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:58.913943    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:45:59.028041 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.028007    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/524851bd-08e5-4413-8e20-4e893631f23e-kube-api-access-8d64g\" (UniqueName: \"kubernetes.io/projected/524851bd-08e5-4413-8e20-4e893631f23e-kube-api-access-8d64g\") pod \"524851bd-08e5-4413-8e20-4e893631f23e\" (UID: \"524851bd-08e5-4413-8e20-4e893631f23e\") "
Nov 02 22:45:59.028242 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.028124    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/524851bd-08e5-4413-8e20-4e893631f23e-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/524851bd-08e5-4413-8e20-4e893631f23e-extensions-socket-volume\") pod \"524851bd-08e5-4413-8e20-4e893631f23e\" (UID: \"524851bd-08e5-4413-8e20-4e893631f23e\") "
Nov 02 22:45:59.028656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.028631    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/524851bd-08e5-4413-8e20-4e893631f23e-extensions-socket-volume" pod "524851bd-08e5-4413-8e20-4e893631f23e" (UID: "524851bd-08e5-4413-8e20-4e893631f23e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:45:59.030499 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.030471    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/524851bd-08e5-4413-8e20-4e893631f23e-kube-api-access-8d64g" pod "524851bd-08e5-4413-8e20-4e893631f23e" (UID: "524851bd-08e5-4413-8e20-4e893631f23e"). InnerVolumeSpecName "kube-api-access-8d64g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:45:59.129585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.129554    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d64g\" (UniqueName: \"kubernetes.io/projected/524851bd-08e5-4413-8e20-4e893631f23e-kube-api-access-8d64g\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:45:59.129585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.129583    5067 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/524851bd-08e5-4413-8e20-4e893631f23e-extensions-socket-volume\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:45:59.703026 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.702986    5067 generic.go:356] "Generic (PLEG): container finished" podID="524851bd-08e5-4413-8e20-4e893631f23e" containerID="038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4" exitCode=0
Nov 02 22:45:59.703495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.703036    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"
Nov 02 22:45:59.703495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.703072    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k" event={"ID":"524851bd-08e5-4413-8e20-4e893631f23e","Type":"ContainerDied","Data":"038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4"}
Nov 02 22:45:59.703495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.703125    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k" event={"ID":"524851bd-08e5-4413-8e20-4e893631f23e","Type":"ContainerDied","Data":"1602db491bc7a44ab5f895d09e0f166d8310fe75a4bb8b8d03ad8da62808bb70"}
Nov 02 22:45:59.703495 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.703146    5067 scope.go:122] "RemoveContainer" containerID="038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4"
Nov 02 22:45:59.716111 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.716093    5067 scope.go:122] "RemoveContainer" containerID="038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4"
Nov 02 22:45:59.716342 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:45:59.716321    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4\": container with ID starting with 038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4 not found: ID does not exist" containerID="038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4"
Nov 02 22:45:59.716444 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.716353    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4"} err="failed to get container status \"038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4\": rpc error: code = NotFound desc = could not find container \"038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4\": container with ID starting with 038549555b0a8f2326206b4a798dc48b56a88fb4d3c2a7e29741fcc668473ee4 not found: ID does not exist"
Nov 02 22:45:59.856952 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.856919    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"]
Nov 02 22:45:59.903526 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:45:59.903494    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mrt2k"]
Nov 02 22:46:01.844252 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:46:01.844213    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="524851bd-08e5-4413-8e20-4e893631f23e" path="/var/lib/kubelet/pods/524851bd-08e5-4413-8e20-4e893631f23e/volumes"
Nov 02 22:46:07.575754 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:46:07.575708    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:46:37.575774 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:46:37.575730    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:47:02.304772 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.304732    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"]
Nov 02 22:47:02.305219 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305201    5067 cpu_manager.go:414] "containerMap: removing container" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerName="cleanup"
Nov 02 22:47:02.305267 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305220    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerName="cleanup"
Nov 02 22:47:02.305267 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305245    5067 cpu_manager.go:414] "containerMap: removing container" podUID="524851bd-08e5-4413-8e20-4e893631f23e" containerName="manager"
Nov 02 22:47:02.305267 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305250    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="524851bd-08e5-4413-8e20-4e893631f23e" containerName="manager"
Nov 02 22:47:02.305267 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305267    5067 cpu_manager.go:414] "containerMap: removing container" podUID="1fdee942-dbbd-476f-85de-82cff84840da" containerName="main"
Nov 02 22:47:02.305436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305273    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fdee942-dbbd-476f-85de-82cff84840da" containerName="main"
Nov 02 22:47:02.305436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305280    5067 cpu_manager.go:414] "containerMap: removing container" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerName="cleanup"
Nov 02 22:47:02.305436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305285    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerName="cleanup"
Nov 02 22:47:02.305436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305340    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerName="cleanup"
Nov 02 22:47:02.305436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305352    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="524851bd-08e5-4413-8e20-4e893631f23e" containerName="manager"
Nov 02 22:47:02.305436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305359    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerName="cleanup"
Nov 02 22:47:02.305436 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.305379    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fdee942-dbbd-476f-85de-82cff84840da" containerName="main"
Nov 02 22:47:02.308248 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.308228    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"
Nov 02 22:47:02.405503 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.405469    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rtphq\""
Nov 02 22:47:02.407296 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.407274    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"]
Nov 02 22:47:02.414917 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.414893    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb79201f-57ac-42df-92b6-751871eebe23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-cbfrn\" (UID: \"bb79201f-57ac-42df-92b6-751871eebe23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"
Nov 02 22:47:02.415029 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.414923    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkbx4\" (UniqueName: \"kubernetes.io/projected/bb79201f-57ac-42df-92b6-751871eebe23-kube-api-access-kkbx4\") pod \"kuadrant-operator-controller-manager-678d87766f-cbfrn\" (UID: \"bb79201f-57ac-42df-92b6-751871eebe23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"
Nov 02 22:47:02.516492 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.516456    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb79201f-57ac-42df-92b6-751871eebe23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-cbfrn\" (UID: \"bb79201f-57ac-42df-92b6-751871eebe23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"
Nov 02 22:47:02.516492 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.516495    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkbx4\" (UniqueName: \"kubernetes.io/projected/bb79201f-57ac-42df-92b6-751871eebe23-kube-api-access-kkbx4\") pod \"kuadrant-operator-controller-manager-678d87766f-cbfrn\" (UID: \"bb79201f-57ac-42df-92b6-751871eebe23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"
Nov 02 22:47:02.516881 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.516862    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb79201f-57ac-42df-92b6-751871eebe23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-cbfrn\" (UID: \"bb79201f-57ac-42df-92b6-751871eebe23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"
Nov 02 22:47:02.574937 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.574863    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkbx4\" (UniqueName: \"kubernetes.io/projected/bb79201f-57ac-42df-92b6-751871eebe23-kube-api-access-kkbx4\") pod \"kuadrant-operator-controller-manager-678d87766f-cbfrn\" (UID: \"bb79201f-57ac-42df-92b6-751871eebe23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"
Nov 02 22:47:02.620475 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:02.620442    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"
Nov 02 22:47:03.084899 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:03.084874    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"]
Nov 02 22:47:03.087806 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:47:03.087772    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb79201f_57ac_42df_92b6_751871eebe23.slice/crio-b91b3993bc46787c6738f03ebaaf5f063a25dc14af318120223eb205f0f1ddce WatchSource:0}: Error finding container b91b3993bc46787c6738f03ebaaf5f063a25dc14af318120223eb205f0f1ddce: Status 404 returned error can't find the container with id b91b3993bc46787c6738f03ebaaf5f063a25dc14af318120223eb205f0f1ddce
Nov 02 22:47:03.996708 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:03.996666    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn" event={"ID":"bb79201f-57ac-42df-92b6-751871eebe23","Type":"ContainerStarted","Data":"123f88766f8a4b9edf1c74ff6501218a75c9f7e088b75a657a9b52b6ef644782"}
Nov 02 22:47:03.997219 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:03.996722    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn" event={"ID":"bb79201f-57ac-42df-92b6-751871eebe23","Type":"ContainerStarted","Data":"b91b3993bc46787c6738f03ebaaf5f063a25dc14af318120223eb205f0f1ddce"}
Nov 02 22:47:03.997219 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:03.996790    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"
Nov 02 22:47:04.157049 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:04.156980    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn" podStartSLOduration=2.156964751 podStartE2EDuration="2.156964751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:47:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:47:04.155398372 +0000 UTC m=+2496.439208060" watchObservedRunningTime="2026-11-02 22:47:04.156964751 +0000 UTC m=+2496.440774438"
Nov 02 22:47:07.578490 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:47:07.578444    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:47:15.004824 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:47:15.004789    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cbfrn"
Nov 02 22:47:37.576250 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:47:37.576210    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:48:07.576160 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:48:07.576112    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:48:37.576840 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:48:37.576791    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:48:46.683268 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.683231    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"]
Nov 02 22:48:46.683708 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.683694    5067 cpu_manager.go:414] "containerMap: removing container" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerName="cleanup"
Nov 02 22:48:46.683760 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.683712    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerName="cleanup"
Nov 02 22:48:46.683796 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.683790    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="43a94e39-5f1a-46ad-9ba4-1cff74c726b1" containerName="cleanup"
Nov 02 22:48:46.687119 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.687100    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.729979 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.729940    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-model-cache\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.730143 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.729997    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5rvx\" (UniqueName: \"kubernetes.io/projected/32bfabc0-6309-454a-8f45-b5f8723956ef-kube-api-access-z5rvx\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.730143 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.730095    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-home\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.730143 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.730119    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32bfabc0-6309-454a-8f45-b5f8723956ef-tls-certs\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.730270 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.730245    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-tmp-dir\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.730322 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.730308    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-dshm\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.736108 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.736079    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-c86709\"/\"openshift-service-ca.crt\""
Nov 02 22:48:46.752784 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.752752    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-c86709\"/\"kube-root-ca.crt\""
Nov 02 22:48:46.782521 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.782489    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-c86709\"/\"default-dockercfg-brcc7\""
Nov 02 22:48:46.782682 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.782664    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-c86709\"/\"simulator-single-main-c86709-kserve-self-signed-certs\""
Nov 02 22:48:46.784461 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.784441    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"]
Nov 02 22:48:46.831258 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.831194    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-model-cache\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.831258 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.831251    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5rvx\" (UniqueName: \"kubernetes.io/projected/32bfabc0-6309-454a-8f45-b5f8723956ef-kube-api-access-z5rvx\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.831641 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.831292    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-home\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.831641 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.831320    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32bfabc0-6309-454a-8f45-b5f8723956ef-tls-certs\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.831641 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.831401    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-tmp-dir\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.831641 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.831453    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-dshm\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.832323 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.832295    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-home\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.832581 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.832561    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-model-cache\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.832980 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.832958    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-tmp-dir\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.837857 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.837830    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-dshm\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.838239 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.838213    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32bfabc0-6309-454a-8f45-b5f8723956ef-tls-certs\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.890129 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.890085    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5rvx\" (UniqueName: \"kubernetes.io/projected/32bfabc0-6309-454a-8f45-b5f8723956ef-kube-api-access-z5rvx\") pod \"simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") " pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:46.998756 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:46.998708    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:47.369408 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:47.369346    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"]
Nov 02 22:48:47.372642 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:48:47.372604    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32bfabc0_6309_454a_8f45_b5f8723956ef.slice/crio-eb27ea9f1160aa8c1cb971fbb47d8b83668e1d025caae6b224b959bf58479fd0 WatchSource:0}: Error finding container eb27ea9f1160aa8c1cb971fbb47d8b83668e1d025caae6b224b959bf58479fd0: Status 404 returned error can't find the container with id eb27ea9f1160aa8c1cb971fbb47d8b83668e1d025caae6b224b959bf58479fd0
Nov 02 22:48:47.603881 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:47.603842    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9" event={"ID":"32bfabc0-6309-454a-8f45-b5f8723956ef","Type":"ContainerStarted","Data":"d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b"}
Nov 02 22:48:47.603881 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:47.603885    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9" event={"ID":"32bfabc0-6309-454a-8f45-b5f8723956ef","Type":"ContainerStarted","Data":"eb27ea9f1160aa8c1cb971fbb47d8b83668e1d025caae6b224b959bf58479fd0"}
Nov 02 22:48:47.713254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:47.713145    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9" podStartSLOduration=1.713130195 podStartE2EDuration="1.713130195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:48:47.712984618 +0000 UTC m=+2599.996794335" watchObservedRunningTime="2026-11-02 22:48:47.713130195 +0000 UTC m=+2599.996939879"
Nov 02 22:48:47.998959 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:47.998919    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:56.999590 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:56.999551    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:57.012230 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:57.012194    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:48:57.676927 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:48:57.676895    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:49:07.575891 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:49:07.575822    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:49:37.578283 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:49:37.578243    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:49:48.602023 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:49:48.601983    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"]
Nov 02 22:50:03.622158 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.622086    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9" podUID="32bfabc0-6309-454a-8f45-b5f8723956ef" containerName="main" containerID="cri-o://d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b" gracePeriod=45
Nov 02 22:50:03.666768 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.666727    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9" podUID="32bfabc0-6309-454a-8f45-b5f8723956ef" containerName="main" probeResult="failure" output="Get \"https://10.132.0.109:8000/ready\": dial tcp 10.132.0.109:8000: connect: connection refused"
Nov 02 22:50:03.874008 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.873947    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:50:03.909264 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909236    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-home\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-home\") pod \"32bfabc0-6309-454a-8f45-b5f8723956ef\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") "
Nov 02 22:50:03.909447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909283    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-tmp-dir\") pod \"32bfabc0-6309-454a-8f45-b5f8723956ef\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") "
Nov 02 22:50:03.909447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909305    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-dshm\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-dshm\") pod \"32bfabc0-6309-454a-8f45-b5f8723956ef\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") "
Nov 02 22:50:03.909447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909331    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32bfabc0-6309-454a-8f45-b5f8723956ef-kube-api-access-z5rvx\" (UniqueName: \"kubernetes.io/projected/32bfabc0-6309-454a-8f45-b5f8723956ef-kube-api-access-z5rvx\") pod \"32bfabc0-6309-454a-8f45-b5f8723956ef\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") "
Nov 02 22:50:03.909447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909384    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-model-cache\") pod \"32bfabc0-6309-454a-8f45-b5f8723956ef\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") "
Nov 02 22:50:03.909447 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909427    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32bfabc0-6309-454a-8f45-b5f8723956ef-tls-certs\" (UniqueName: \"kubernetes.io/secret/32bfabc0-6309-454a-8f45-b5f8723956ef-tls-certs\") pod \"32bfabc0-6309-454a-8f45-b5f8723956ef\" (UID: \"32bfabc0-6309-454a-8f45-b5f8723956ef\") "
Nov 02 22:50:03.909724 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909525    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-home" pod "32bfabc0-6309-454a-8f45-b5f8723956ef" (UID: "32bfabc0-6309-454a-8f45-b5f8723956ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:50:03.909724 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909563    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-tmp-dir" pod "32bfabc0-6309-454a-8f45-b5f8723956ef" (UID: "32bfabc0-6309-454a-8f45-b5f8723956ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:50:03.909724 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909657    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-model-cache" pod "32bfabc0-6309-454a-8f45-b5f8723956ef" (UID: "32bfabc0-6309-454a-8f45-b5f8723956ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:50:03.909899 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909831    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:50:03.909899 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909852    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:50:03.909899 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.909880    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:50:03.911676 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.911655    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-dshm" pod "32bfabc0-6309-454a-8f45-b5f8723956ef" (UID: "32bfabc0-6309-454a-8f45-b5f8723956ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:50:03.911760 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.911689    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32bfabc0-6309-454a-8f45-b5f8723956ef-kube-api-access-z5rvx" pod "32bfabc0-6309-454a-8f45-b5f8723956ef" (UID: "32bfabc0-6309-454a-8f45-b5f8723956ef"). InnerVolumeSpecName "kube-api-access-z5rvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:50:03.911760 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:03.911742    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32bfabc0-6309-454a-8f45-b5f8723956ef-tls-certs" pod "32bfabc0-6309-454a-8f45-b5f8723956ef" (UID: "32bfabc0-6309-454a-8f45-b5f8723956ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:50:04.006016 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.005977    5067 generic.go:356] "Generic (PLEG): container finished" podID="32bfabc0-6309-454a-8f45-b5f8723956ef" containerID="d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b" exitCode=0
Nov 02 22:50:04.006016 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.006025    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"
Nov 02 22:50:04.006226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.006062    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9" event={"ID":"32bfabc0-6309-454a-8f45-b5f8723956ef","Type":"ContainerDied","Data":"d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b"}
Nov 02 22:50:04.006226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.006111    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9" event={"ID":"32bfabc0-6309-454a-8f45-b5f8723956ef","Type":"ContainerDied","Data":"eb27ea9f1160aa8c1cb971fbb47d8b83668e1d025caae6b224b959bf58479fd0"}
Nov 02 22:50:04.006226 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.006133    5067 scope.go:122] "RemoveContainer" containerID="d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b"
Nov 02 22:50:04.010970 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.010950    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32bfabc0-6309-454a-8f45-b5f8723956ef-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:50:04.011070 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.010975    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5rvx\" (UniqueName: \"kubernetes.io/projected/32bfabc0-6309-454a-8f45-b5f8723956ef-kube-api-access-z5rvx\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:50:04.011070 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.010995    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32bfabc0-6309-454a-8f45-b5f8723956ef-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:50:04.017476 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.017451    5067 scope.go:122] "RemoveContainer" containerID="d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b"
Nov 02 22:50:04.017798 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:50:04.017771    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b\": container with ID starting with d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b not found: ID does not exist" containerID="d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b"
Nov 02 22:50:04.017894 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.017808    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b"} err="failed to get container status \"d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b\": rpc error: code = NotFound desc = could not find container \"d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b\": container with ID starting with d60eb56d3062645eb211eedcb2a06f473279c03a0a7b7ce1cd131d4b8321a34b not found: ID does not exist"
Nov 02 22:50:04.159896 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.159808    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"]
Nov 02 22:50:04.163998 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:04.163964    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-c86709/simulator-single-main-c86709-kserve-8fd48c7bf-2jsc9"]
Nov 02 22:50:05.844273 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:05.844238    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32bfabc0-6309-454a-8f45-b5f8723956ef" path="/var/lib/kubelet/pods/32bfabc0-6309-454a-8f45-b5f8723956ef/volumes"
Nov 02 22:50:07.577442 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:50:07.577403    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:50:28.732113 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:28.732084    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:50:28.732623 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:28.732153    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:50:37.576453 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:50:37.576413    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:50:53.773449 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.773408    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"]
Nov 02 22:50:53.773997 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.773976    5067 cpu_manager.go:414] "containerMap: removing container" podUID="32bfabc0-6309-454a-8f45-b5f8723956ef" containerName="main"
Nov 02 22:50:53.774084 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.774009    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32bfabc0-6309-454a-8f45-b5f8723956ef" containerName="main"
Nov 02 22:50:53.774182 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.774169    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="32bfabc0-6309-454a-8f45-b5f8723956ef" containerName="main"
Nov 02 22:50:53.778233 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.778209    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.870649 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.870615    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-model-cache\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.870820 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.870737    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-home\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.870820 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.870784    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7g6j\" (UniqueName: \"kubernetes.io/projected/d86588d3-4203-44ff-a781-e0e17ba9123d-kube-api-access-c7g6j\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.870820 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.870805    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-tmp-dir\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.870983 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.870829    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-dshm\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.870983 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.870857    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d86588d3-4203-44ff-a781-e0e17ba9123d-tls-certs\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.876541 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.876517    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 02 22:50:53.876785 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.876768    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-swvkh\""
Nov 02 22:50:53.876862 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.876835    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 02 22:50:53.876953 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.876894    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e2817aa1-kserve-self-signed-certs\""
Nov 02 22:50:53.878526 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.878502    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"]
Nov 02 22:50:53.972038 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.971993    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d86588d3-4203-44ff-a781-e0e17ba9123d-tls-certs\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.972245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.972082    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-model-cache\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.972245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.972161    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-home\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.972245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.972216    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7g6j\" (UniqueName: \"kubernetes.io/projected/d86588d3-4203-44ff-a781-e0e17ba9123d-kube-api-access-c7g6j\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.972245 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.972242    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-tmp-dir\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.972526 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.972279    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-dshm\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.972656 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.972613    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-model-cache\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.972746 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.972668    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-home\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.972867 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.972846    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-tmp-dir\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.974625 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.974601    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-dshm\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:53.974765 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:53.974734    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d86588d3-4203-44ff-a781-e0e17ba9123d-tls-certs\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:54.033017 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:54.032945    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7g6j\" (UniqueName: \"kubernetes.io/projected/d86588d3-4203-44ff-a781-e0e17ba9123d-kube-api-access-c7g6j\") pod \"e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") " pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:54.089797 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:54.089761    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:50:54.472404 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:54.472351    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"]
Nov 02 22:50:54.475531 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:50:54.475502    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd86588d3_4203_44ff_a781_e0e17ba9123d.slice/crio-51bdd9b1047a76fb2ae892cc7b1501c330a084ed6b91d1a946ba55204392f9a9 WatchSource:0}: Error finding container 51bdd9b1047a76fb2ae892cc7b1501c330a084ed6b91d1a946ba55204392f9a9: Status 404 returned error can't find the container with id 51bdd9b1047a76fb2ae892cc7b1501c330a084ed6b91d1a946ba55204392f9a9
Nov 02 22:50:55.287685 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:55.287651    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch" event={"ID":"d86588d3-4203-44ff-a781-e0e17ba9123d","Type":"ContainerStarted","Data":"81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c"}
Nov 02 22:50:55.288096 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:55.287694    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch" event={"ID":"d86588d3-4203-44ff-a781-e0e17ba9123d","Type":"ContainerStarted","Data":"51bdd9b1047a76fb2ae892cc7b1501c330a084ed6b91d1a946ba55204392f9a9"}
Nov 02 22:50:55.400884 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:55.400807    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch" podStartSLOduration=2.400790315 podStartE2EDuration="2.400790315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:50:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:50:55.397196826 +0000 UTC m=+2727.681006515" watchObservedRunningTime="2026-11-02 22:50:55.400790315 +0000 UTC m=+2727.684600003"
Nov 02 22:50:56.090382 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:50:56.090336    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:51:04.090810 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:51:04.090773    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:51:04.103811 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:51:04.103781    5067 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:51:04.393077 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:51:04.392988    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:51:07.577250 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:51:07.577210    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:51:37.576499 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:51:37.576450    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:51:46.762395 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:51:46.762344    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"]
Nov 02 22:52:01.782448 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:01.782378    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch" podUID="d86588d3-4203-44ff-a781-e0e17ba9123d" containerName="main" containerID="cri-o://81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c" gracePeriod=45
Nov 02 22:52:02.049854 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.049821    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:52:02.142733 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.142688    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-home\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-home\") pod \"d86588d3-4203-44ff-a781-e0e17ba9123d\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") "
Nov 02 22:52:02.142904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.142805    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d86588d3-4203-44ff-a781-e0e17ba9123d-tls-certs\" (UniqueName: \"kubernetes.io/secret/d86588d3-4203-44ff-a781-e0e17ba9123d-tls-certs\") pod \"d86588d3-4203-44ff-a781-e0e17ba9123d\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") "
Nov 02 22:52:02.142904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.142860    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-model-cache\") pod \"d86588d3-4203-44ff-a781-e0e17ba9123d\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") "
Nov 02 22:52:02.142904 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.142875    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-home" pod "d86588d3-4203-44ff-a781-e0e17ba9123d" (UID: "d86588d3-4203-44ff-a781-e0e17ba9123d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:52:02.143043 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.142913    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-dshm\") pod \"d86588d3-4203-44ff-a781-e0e17ba9123d\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") "
Nov 02 22:52:02.143043 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.142950    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-tmp-dir\") pod \"d86588d3-4203-44ff-a781-e0e17ba9123d\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") "
Nov 02 22:52:02.143043 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.142982    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d86588d3-4203-44ff-a781-e0e17ba9123d-kube-api-access-c7g6j\" (UniqueName: \"kubernetes.io/projected/d86588d3-4203-44ff-a781-e0e17ba9123d-kube-api-access-c7g6j\") pod \"d86588d3-4203-44ff-a781-e0e17ba9123d\" (UID: \"d86588d3-4203-44ff-a781-e0e17ba9123d\") "
Nov 02 22:52:02.143203 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.143103    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-model-cache" pod "d86588d3-4203-44ff-a781-e0e17ba9123d" (UID: "d86588d3-4203-44ff-a781-e0e17ba9123d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:52:02.143265 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.143207    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-tmp-dir" pod "d86588d3-4203-44ff-a781-e0e17ba9123d" (UID: "d86588d3-4203-44ff-a781-e0e17ba9123d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:52:02.143409 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.143389    5067 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-home\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:52:02.143486 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.143413    5067 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-model-cache\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:52:02.143486 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.143427    5067 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-tmp-dir\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:52:02.145143 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.145118    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d86588d3-4203-44ff-a781-e0e17ba9123d-tls-certs" pod "d86588d3-4203-44ff-a781-e0e17ba9123d" (UID: "d86588d3-4203-44ff-a781-e0e17ba9123d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:52:02.145259 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.145223    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-dshm" pod "d86588d3-4203-44ff-a781-e0e17ba9123d" (UID: "d86588d3-4203-44ff-a781-e0e17ba9123d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 22:52:02.145342 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.145318    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d86588d3-4203-44ff-a781-e0e17ba9123d-kube-api-access-c7g6j" pod "d86588d3-4203-44ff-a781-e0e17ba9123d" (UID: "d86588d3-4203-44ff-a781-e0e17ba9123d"). InnerVolumeSpecName "kube-api-access-c7g6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:52:02.244638 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.244602    5067 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d86588d3-4203-44ff-a781-e0e17ba9123d-dshm\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:52:02.244638 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.244633    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7g6j\" (UniqueName: \"kubernetes.io/projected/d86588d3-4203-44ff-a781-e0e17ba9123d-kube-api-access-c7g6j\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:52:02.244638 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.244644    5067 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d86588d3-4203-44ff-a781-e0e17ba9123d-tls-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:52:02.844666 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.844632    5067 generic.go:356] "Generic (PLEG): container finished" podID="d86588d3-4203-44ff-a781-e0e17ba9123d" containerID="81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c" exitCode=0
Nov 02 22:52:02.845102 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.844682    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"
Nov 02 22:52:02.845102 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.844713    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch" event={"ID":"d86588d3-4203-44ff-a781-e0e17ba9123d","Type":"ContainerDied","Data":"81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c"}
Nov 02 22:52:02.845102 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.844755    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch" event={"ID":"d86588d3-4203-44ff-a781-e0e17ba9123d","Type":"ContainerDied","Data":"51bdd9b1047a76fb2ae892cc7b1501c330a084ed6b91d1a946ba55204392f9a9"}
Nov 02 22:52:02.845102 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.844773    5067 scope.go:122] "RemoveContainer" containerID="81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c"
Nov 02 22:52:02.854342 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.854318    5067 scope.go:122] "RemoveContainer" containerID="81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c"
Nov 02 22:52:02.854627 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:52:02.854600    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c\": container with ID starting with 81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c not found: ID does not exist" containerID="81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c"
Nov 02 22:52:02.854701 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.854635    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c"} err="failed to get container status \"81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c\": rpc error: code = NotFound desc = could not find container \"81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c\": container with ID starting with 81783937a03fdf1377d51e72ea4c6620bee546ce9c451acb2e57e36d54436a1c not found: ID does not exist"
Nov 02 22:52:02.998960 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:02.998927    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"]
Nov 02 22:52:03.045340 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:03.045308    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e2817aa1-kserve-8f86c997c-8qjch"]
Nov 02 22:52:03.844230 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:03.844191    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d86588d3-4203-44ff-a781-e0e17ba9123d" path="/var/lib/kubelet/pods/d86588d3-4203-44ff-a781-e0e17ba9123d/volumes"
Nov 02 22:52:07.748932 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:52:07.748846    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:52:33.927842 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:33.927806    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"]
Nov 02 22:52:33.928422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:33.928241    5067 cpu_manager.go:414] "containerMap: removing container" podUID="d86588d3-4203-44ff-a781-e0e17ba9123d" containerName="main"
Nov 02 22:52:33.928422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:33.928256    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d86588d3-4203-44ff-a781-e0e17ba9123d" containerName="main"
Nov 02 22:52:33.928422 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:33.928344    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="d86588d3-4203-44ff-a781-e0e17ba9123d" containerName="main"
Nov 02 22:52:33.933972 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:33.933948    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:33.955278 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:33.955250    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dkhb\" (UniqueName: \"kubernetes.io/projected/f138ed50-af8c-448e-a58f-8c64cb0bf05d-kube-api-access-7dkhb\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:33.955437 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:33.955359    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-metrics-certs\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:33.955437 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:33.955427    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f138ed50-af8c-448e-a58f-8c64cb0bf05d-openshift-service-ca\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:33.955611 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:33.955570    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-maas-api-tls\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:34.004212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.004182    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 22:52:34.004212 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.004191    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 22:52:34.004460 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.004268    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-l4djg\""
Nov 02 22:52:34.029455 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.029419    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 02 22:52:34.029455 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.029444    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 22:52:34.031828 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.031804    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"]
Nov 02 22:52:34.056381 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.056338    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-metrics-certs\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:34.056554 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.056428    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f138ed50-af8c-448e-a58f-8c64cb0bf05d-openshift-service-ca\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:34.056554 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.056488    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-maas-api-tls\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:34.056554 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.056539    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dkhb\" (UniqueName: \"kubernetes.io/projected/f138ed50-af8c-448e-a58f-8c64cb0bf05d-kube-api-access-7dkhb\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:34.057254 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.057220    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f138ed50-af8c-448e-a58f-8c64cb0bf05d-openshift-service-ca\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:34.059063 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.059042    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-metrics-certs\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:34.059294 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.059271    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-maas-api-tls\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:34.114641 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.114610    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dkhb\" (UniqueName: \"kubernetes.io/projected/f138ed50-af8c-448e-a58f-8c64cb0bf05d-kube-api-access-7dkhb\") pod \"maas-api-5bb6546495-szj6q\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") " pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:34.247308 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.247267    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:34.620081 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.620000    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"]
Nov 02 22:52:34.623167 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:52:34.623134    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf138ed50_af8c_448e_a58f_8c64cb0bf05d.slice/crio-a539cb3301f9ee01b7f24e94693751c7ee00d84290da87f4ca7dba8207c7c55b WatchSource:0}: Error finding container a539cb3301f9ee01b7f24e94693751c7ee00d84290da87f4ca7dba8207c7c55b: Status 404 returned error can't find the container with id a539cb3301f9ee01b7f24e94693751c7ee00d84290da87f4ca7dba8207c7c55b
Nov 02 22:52:34.624574 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:34.624557    5067 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 22:52:35.076295 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:35.076259    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q" event={"ID":"f138ed50-af8c-448e-a58f-8c64cb0bf05d","Type":"ContainerStarted","Data":"a539cb3301f9ee01b7f24e94693751c7ee00d84290da87f4ca7dba8207c7c55b"}
Nov 02 22:52:36.082508 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:36.082466    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q" event={"ID":"f138ed50-af8c-448e-a58f-8c64cb0bf05d","Type":"ContainerStarted","Data":"d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465"}
Nov 02 22:52:36.082977 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:36.082581    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:36.195557 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:36.195499    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q" podStartSLOduration=2.193675601 podStartE2EDuration="3.195485521s" totalImagesPullingTime="1.00180992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:52:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:52:36.193616838 +0000 UTC m=+2828.477426544" watchObservedRunningTime="2026-11-02 22:52:36.195485521 +0000 UTC m=+2828.479295208"
Nov 02 22:52:37.582509 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:52:37.582102    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:52:42.094033 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.094003    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:42.287560 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.287522    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"]
Nov 02 22:52:42.291295 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.291263    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.346355 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.346268    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"]
Nov 02 22:52:42.347600 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.347570    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqpnf\" (UniqueName: \"kubernetes.io/projected/c51c558e-4428-46a4-b260-bae8d37e17c1-kube-api-access-mqpnf\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.347740 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.347629    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51c558e-4428-46a4-b260-bae8d37e17c1-openshift-service-ca\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.347832 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.347808    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c51c558e-4428-46a4-b260-bae8d37e17c1-maas-api-tls\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.347896 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.347869    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c51c558e-4428-46a4-b260-bae8d37e17c1-metrics-certs\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.448917 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.448872    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c51c558e-4428-46a4-b260-bae8d37e17c1-metrics-certs\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.449146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.449018    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqpnf\" (UniqueName: \"kubernetes.io/projected/c51c558e-4428-46a4-b260-bae8d37e17c1-kube-api-access-mqpnf\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.449146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.449063    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51c558e-4428-46a4-b260-bae8d37e17c1-openshift-service-ca\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.449146 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.449105    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c51c558e-4428-46a4-b260-bae8d37e17c1-maas-api-tls\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.449885 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.449851    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c51c558e-4428-46a4-b260-bae8d37e17c1-openshift-service-ca\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.451665 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.451632    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c51c558e-4428-46a4-b260-bae8d37e17c1-maas-api-tls\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.451773 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.451704    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c51c558e-4428-46a4-b260-bae8d37e17c1-metrics-certs\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.506425 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.506390    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqpnf\" (UniqueName: \"kubernetes.io/projected/c51c558e-4428-46a4-b260-bae8d37e17c1-kube-api-access-mqpnf\") pod \"maas-api-86c65979d7-j5zvc\" (UID: \"c51c558e-4428-46a4-b260-bae8d37e17c1\") " pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.607127 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.607033    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:42.976466 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:42.976432    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"]
Nov 02 22:52:42.979949 ip-10-0-130-41 kubenswrapper[5067]: W1102 22:52:42.979921    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51c558e_4428_46a4_b260_bae8d37e17c1.slice/crio-91b3c4530c9687563db904eaa55d7a8f79525b3cc5d7d4a9522b808a322b2f4d WatchSource:0}: Error finding container 91b3c4530c9687563db904eaa55d7a8f79525b3cc5d7d4a9522b808a322b2f4d: Status 404 returned error can't find the container with id 91b3c4530c9687563db904eaa55d7a8f79525b3cc5d7d4a9522b808a322b2f4d
Nov 02 22:52:43.117247 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:43.117207    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc" event={"ID":"c51c558e-4428-46a4-b260-bae8d37e17c1","Type":"ContainerStarted","Data":"91b3c4530c9687563db904eaa55d7a8f79525b3cc5d7d4a9522b808a322b2f4d"}
Nov 02 22:52:44.124576 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:44.124534    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc" event={"ID":"c51c558e-4428-46a4-b260-bae8d37e17c1","Type":"ContainerStarted","Data":"293d1cdf9e273a02feb459bc086c9a9534895117b026d1118e4d5b0084322bbe"}
Nov 02 22:52:44.124996 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:44.124607    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:44.238301 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:44.238245    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc" podStartSLOduration=1.284028088 podStartE2EDuration="2.238230305s" totalImagesPullingTime="954.202217ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 22:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 22:52:44.235537819 +0000 UTC m=+2836.519347528" watchObservedRunningTime="2026-11-02 22:52:44.238230305 +0000 UTC m=+2836.522039991"
Nov 02 22:52:50.140265 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.140234    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86c65979d7-j5zvc"
Nov 02 22:52:50.301598 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.301557    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"]
Nov 02 22:52:50.301853 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.301825    5067 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q" podUID="f138ed50-af8c-448e-a58f-8c64cb0bf05d" containerName="maas-api" containerID="cri-o://d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465" gracePeriod=30
Nov 02 22:52:50.562159 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.562132    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:50.631271 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.631238    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-maas-api-tls\") pod \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") "
Nov 02 22:52:50.631458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.631305    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f138ed50-af8c-448e-a58f-8c64cb0bf05d-kube-api-access-7dkhb\" (UniqueName: \"kubernetes.io/projected/f138ed50-af8c-448e-a58f-8c64cb0bf05d-kube-api-access-7dkhb\") pod \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") "
Nov 02 22:52:50.631458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.631387    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f138ed50-af8c-448e-a58f-8c64cb0bf05d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f138ed50-af8c-448e-a58f-8c64cb0bf05d-openshift-service-ca\") pod \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") "
Nov 02 22:52:50.631458 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.631443    5067 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-metrics-certs\") pod \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\" (UID: \"f138ed50-af8c-448e-a58f-8c64cb0bf05d\") "
Nov 02 22:52:50.631827 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.631789    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f138ed50-af8c-448e-a58f-8c64cb0bf05d-openshift-service-ca" pod "f138ed50-af8c-448e-a58f-8c64cb0bf05d" (UID: "f138ed50-af8c-448e-a58f-8c64cb0bf05d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 22:52:50.633573 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.633544    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-metrics-certs" pod "f138ed50-af8c-448e-a58f-8c64cb0bf05d" (UID: "f138ed50-af8c-448e-a58f-8c64cb0bf05d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:52:50.633664 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.633577    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f138ed50-af8c-448e-a58f-8c64cb0bf05d-kube-api-access-7dkhb" pod "f138ed50-af8c-448e-a58f-8c64cb0bf05d" (UID: "f138ed50-af8c-448e-a58f-8c64cb0bf05d"). InnerVolumeSpecName "kube-api-access-7dkhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 22:52:50.633664 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.633577    5067 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-maas-api-tls" pod "f138ed50-af8c-448e-a58f-8c64cb0bf05d" (UID: "f138ed50-af8c-448e-a58f-8c64cb0bf05d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 22:52:50.732870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.732836    5067 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-metrics-certs\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:52:50.732870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.732866    5067 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f138ed50-af8c-448e-a58f-8c64cb0bf05d-maas-api-tls\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:52:50.732870 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.732876    5067 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dkhb\" (UniqueName: \"kubernetes.io/projected/f138ed50-af8c-448e-a58f-8c64cb0bf05d-kube-api-access-7dkhb\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:52:50.733115 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:50.732887    5067 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f138ed50-af8c-448e-a58f-8c64cb0bf05d-openshift-service-ca\") on node \"ip-10-0-130-41.us-west-2.compute.internal\" DevicePath \"\""
Nov 02 22:52:51.163864 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:51.163781    5067 generic.go:356] "Generic (PLEG): container finished" podID="f138ed50-af8c-448e-a58f-8c64cb0bf05d" containerID="d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465" exitCode=0
Nov 02 22:52:51.163864 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:51.163841    5067 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"
Nov 02 22:52:51.164294 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:51.163867    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q" event={"ID":"f138ed50-af8c-448e-a58f-8c64cb0bf05d","Type":"ContainerDied","Data":"d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465"}
Nov 02 22:52:51.164294 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:51.163947    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bb6546495-szj6q" event={"ID":"f138ed50-af8c-448e-a58f-8c64cb0bf05d","Type":"ContainerDied","Data":"a539cb3301f9ee01b7f24e94693751c7ee00d84290da87f4ca7dba8207c7c55b"}
Nov 02 22:52:51.164294 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:51.163969    5067 scope.go:122] "RemoveContainer" containerID="d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465"
Nov 02 22:52:51.174227 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:51.174211    5067 scope.go:122] "RemoveContainer" containerID="d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465"
Nov 02 22:52:51.174530 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:52:51.174510    5067 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465\": container with ID starting with d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465 not found: ID does not exist" containerID="d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465"
Nov 02 22:52:51.174585 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:51.174540    5067 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465"} err="failed to get container status \"d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465\": rpc error: code = NotFound desc = could not find container \"d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465\": container with ID starting with d57574f323f0784700e24a96fd6121d0ae59af673f87b11f5ea04e80770e6465 not found: ID does not exist"
Nov 02 22:52:51.317755 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:51.317718    5067 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"]
Nov 02 22:52:51.364022 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:51.363989    5067 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb6546495-szj6q"]
Nov 02 22:52:51.843670 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:52:51.843639    5067 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f138ed50-af8c-448e-a58f-8c64cb0bf05d" path="/var/lib/kubelet/pods/f138ed50-af8c-448e-a58f-8c64cb0bf05d/volumes"
Nov 02 22:53:07.577386 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:53:07.577337    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:53:37.575493 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:53:37.575454    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:54:07.579870 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:54:07.579824    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:54:37.576634 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:54:37.576586    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:55:07.576162 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:55:07.576120    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:55:28.766241 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:55:28.766204    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:55:28.768408 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:55:28.768384    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 22:55:37.576747 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:55:37.576707    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:56:07.576927 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:56:07.576885    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:56:21.786913 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:56:21.786860    5067 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal" podUID="376e7bd75c236d364d4749b5f447d5f4" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:56:37.591965 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:56:37.591917    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:57:07.576405 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:57:07.576349    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:57:26.677018 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:57:26.676473    5067 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-76f4876667-mxjzx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 22:57:26.677018 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:57:26.676555    5067 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-76f4876667-mxjzx" podUID="f7a52edf-7fef-427a-99cd-4add52eaf196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 22:57:37.575482 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:57:37.575445    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:57:59.339410 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:57:59.339353    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc8f7b988-ccqln_36d566dd-6bc5-428c-ac92-ed9944c78c72/authorino/0.log"
Nov 02 22:58:07.576850 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:58:07.576802    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:58:20.872071 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:20.872040    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pjd76_6f703922-4ab8-4d37-b8dc-03cbbf0eea31/manager/0.log"
Nov 02 22:58:23.046180 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:23.046143    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-86c65979d7-j5zvc_c51c558e-4428-46a4-b260-bae8d37e17c1/maas-api/0.log"
Nov 02 22:58:25.826930 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:25.826898    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc8f7b988-ccqln_36d566dd-6bc5-428c-ac92-ed9944c78c72/authorino/0.log"
Nov 02 22:58:26.616141 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:26.616109    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-hd976_de500486-6d58-48e7-80e3-81b3ff9cbbef/manager/0.log"
Nov 02 22:58:27.122929 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:27.122887    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-cbfrn_bb79201f-57ac-42df-92b6-751871eebe23/manager/0.log"
Nov 02 22:58:28.609603 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:28.609559    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-67c76445c-rm6gt_ec35ca20-d658-4f8a-8725-4d5cd3b5ad87/istio-proxy/0.log"
Nov 02 22:58:29.385614 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:29.385580    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f985b8d7-ph8cc_9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51/kube-auth-proxy/0.log"
Nov 02 22:58:29.619122 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:29.619090    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76f4876667-4jg6k_1d4d7818-f4be-42f8-9db2-28dcbf96091d/istio-proxy/0.log"
Nov 02 22:58:30.394488 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:30.394453    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76f4876667-fzph7_790b1bdf-449a-4671-b640-97990dd0d405/istio-proxy/0.log"
Nov 02 22:58:30.942350 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:30.942319    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76f4876667-m5kv8_72c6a764-cb40-417c-ae95-bde8fd50e29e/istio-proxy/0.log"
Nov 02 22:58:31.232851 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:31.232822    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76f4876667-mxjzx_f7a52edf-7fef-427a-99cd-4add52eaf196/istio-proxy/0.log"
Nov 02 22:58:32.203696 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:32.203659    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2_49754aeb-a62e-4c38-bfb0-75dbc9f12c40/istio-proxy/0.log"
Nov 02 22:58:32.918712 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:32.918682    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b448bfb57-lttxh_dde46c9c-2501-4545-918c-9a3af3a6b381/router/0.log"
Nov 02 22:58:37.575682 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:58:37.575645    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:58:46.340417 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:46.340353    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc8f7b988-ccqln_36d566dd-6bc5-428c-ac92-ed9944c78c72/authorino/0.log"
Nov 02 22:58:56.517903 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:56.517857    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-pjd76_6f703922-4ab8-4d37-b8dc-03cbbf0eea31/manager/0.log"
Nov 02 22:58:58.273134 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:58:58.273103    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-86c65979d7-j5zvc_c51c558e-4428-46a4-b260-bae8d37e17c1/maas-api/0.log"
Nov 02 22:59:00.608133 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:00.608100    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc8f7b988-ccqln_36d566dd-6bc5-428c-ac92-ed9944c78c72/authorino/0.log"
Nov 02 22:59:01.418734 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:01.418692    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-hd976_de500486-6d58-48e7-80e3-81b3ff9cbbef/manager/0.log"
Nov 02 22:59:01.863948 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:01.863910    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-cbfrn_bb79201f-57ac-42df-92b6-751871eebe23/manager/0.log"
Nov 02 22:59:03.169649 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:03.169618    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-67c76445c-rm6gt_ec35ca20-d658-4f8a-8725-4d5cd3b5ad87/istio-proxy/0.log"
Nov 02 22:59:03.922599 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:03.922563    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f985b8d7-ph8cc_9c3fdd6f-e83a-47a3-8d54-576cc6c2ba51/kube-auth-proxy/0.log"
Nov 02 22:59:07.577085 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:59:07.576999    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 22:59:14.163189 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:14.163154    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76f4876667-4jg6k_1d4d7818-f4be-42f8-9db2-28dcbf96091d/istio-proxy/0.log"
Nov 02 22:59:15.005195 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:15.005159    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76f4876667-fzph7_790b1bdf-449a-4671-b640-97990dd0d405/istio-proxy/0.log"
Nov 02 22:59:15.615291 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:15.615263    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76f4876667-m5kv8_72c6a764-cb40-417c-ae95-bde8fd50e29e/istio-proxy/0.log"
Nov 02 22:59:15.905752 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:15.905664    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-76f4876667-mxjzx_f7a52edf-7fef-427a-99cd-4add52eaf196/istio-proxy/0.log"
Nov 02 22:59:16.934986 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:16.934953    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-b6bf76947-kfjm2_49754aeb-a62e-4c38-bfb0-75dbc9f12c40/istio-proxy/0.log"
Nov 02 22:59:17.664794 ip-10-0-130-41 kubenswrapper[5067]: I1102 22:59:17.664761    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b448bfb57-lttxh_dde46c9c-2501-4545-918c-9a3af3a6b381/router/0.log"
Nov 02 22:59:37.576258 ip-10-0-130-41 kubenswrapper[5067]: E1102 22:59:37.576213    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 23:00:07.576577 ip-10-0-130-41 kubenswrapper[5067]: E1102 23:00:07.576537    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 23:00:28.805295 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:00:28.805265    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 23:00:28.807808 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:00:28.807784    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-b5mdw_8f32e923-7f5f-4ba7-b44d-6441fc46f876/console-operator/1.log"
Nov 02 23:00:37.575303 ip-10-0-130-41 kubenswrapper[5067]: E1102 23:00:37.575264    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 23:01:07.578019 ip-10-0-130-41 kubenswrapper[5067]: E1102 23:01:07.577974    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 23:01:37.575915 ip-10-0-130-41 kubenswrapper[5067]: E1102 23:01:37.575874    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 23:01:52.667301 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:01:52.667268    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-78snq_e47f56ea-3a73-4ff0-965d-3a21319aef2e/global-pull-secret-syncer/0.log"
Nov 02 23:01:54.406581 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:01:54.406541    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-7vddh_012f51ba-d2ba-4d6b-b90d-67fe89601119/connection-checker/0.log"
Nov 02 23:01:54.820956 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:01:54.820929    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-tjhsr_101f50b9-bfd1-405a-ade4-6e1cc683dfda/connection-checker/0.log"
Nov 02 23:01:55.231541 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:01:55.231511    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-xmv5g_34ca0f75-5a97-4496-a7ae-060723b2d625/connection-checker/0.log"
Nov 02 23:01:56.078741 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:01:56.078711    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ww4zl_4843105e-92f4-4b83-83fd-06e7a217e009/konnectivity-agent/0.log"
Nov 02 23:01:57.046966 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:01:57.046935    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-41.us-west-2.compute.internal_376e7bd75c236d364d4749b5f447d5f4/haproxy/0.log"
Nov 02 23:02:02.624037 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:02.624006    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dc8f7b988-ccqln_36d566dd-6bc5-428c-ac92-ed9944c78c72/authorino/0.log"
Nov 02 23:02:03.373267 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:03.373234    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-hd976_de500486-6d58-48e7-80e3-81b3ff9cbbef/manager/0.log"
Nov 02 23:02:03.949698 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:03.949663    5067 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"]
Nov 02 23:02:03.950100 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:03.950087    5067 cpu_manager.go:414] "containerMap: removing container" podUID="f138ed50-af8c-448e-a58f-8c64cb0bf05d" containerName="maas-api"
Nov 02 23:02:03.950148 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:03.950101    5067 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f138ed50-af8c-448e-a58f-8c64cb0bf05d" containerName="maas-api"
Nov 02 23:02:03.950184 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:03.950180    5067 memory_manager.go:365] "RemoveStaleState removing state" podUID="f138ed50-af8c-448e-a58f-8c64cb0bf05d" containerName="maas-api"
Nov 02 23:02:03.953239 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:03.953221    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.035698 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.035659    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"]
Nov 02 23:02:04.047321 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.047287    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-proc\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.047501 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.047330    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4kzt\" (UniqueName: \"kubernetes.io/projected/42f029e5-9d90-4e67-81ba-ed390a6d05af-kube-api-access-w4kzt\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.047501 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.047462    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-podres\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.047501 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.047491    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-sys\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.047632 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.047571    5067 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-lib-modules\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.091291 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.091260    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-drfxk\"/\"kube-root-ca.crt\""
Nov 02 23:02:04.105964 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.105932    5067 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-drfxk\"/\"openshift-service-ca.crt\""
Nov 02 23:02:04.105964 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.105953    5067 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-drfxk\"/\"default-dockercfg-x5w7p\""
Nov 02 23:02:04.149195 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.149150    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-proc\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.149195 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.149197    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4kzt\" (UniqueName: \"kubernetes.io/projected/42f029e5-9d90-4e67-81ba-ed390a6d05af-kube-api-access-w4kzt\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.149484 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.149292    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-proc\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.149484 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.149411    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-podres\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.149484 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.149442    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-sys\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.149658 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.149519    5067 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-lib-modules\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.149658 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.149547    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-podres\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.149658 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.149578    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-sys\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.149658 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.149627    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42f029e5-9d90-4e67-81ba-ed390a6d05af-lib-modules\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.207188 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.207111    5067 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4kzt\" (UniqueName: \"kubernetes.io/projected/42f029e5-9d90-4e67-81ba-ed390a6d05af-kube-api-access-w4kzt\") pod \"perf-node-gather-daemonset-4ngt8\" (UID: \"42f029e5-9d90-4e67-81ba-ed390a6d05af\") " pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.239197 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.239164    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-cbfrn_bb79201f-57ac-42df-92b6-751871eebe23/manager/0.log"
Nov 02 23:02:04.264976 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.264949    5067 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:04.641596 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.641565    5067 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"]
Nov 02 23:02:04.644554 ip-10-0-130-41 kubenswrapper[5067]: W1102 23:02:04.644524    5067 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod42f029e5_9d90_4e67_81ba_ed390a6d05af.slice/crio-81ceac15144450fddf43b924ffb94f096d4a7c09f595198376fb869b77ead65d WatchSource:0}: Error finding container 81ceac15144450fddf43b924ffb94f096d4a7c09f595198376fb869b77ead65d: Status 404 returned error can't find the container with id 81ceac15144450fddf43b924ffb94f096d4a7c09f595198376fb869b77ead65d
Nov 02 23:02:04.978406 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.978346    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8" event={"ID":"42f029e5-9d90-4e67-81ba-ed390a6d05af","Type":"ContainerStarted","Data":"7f84f202431cbc26974afa9e74cf7fafeb174055cae264b4ca64200d32724040"}
Nov 02 23:02:04.978817 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.978434    5067 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8" event={"ID":"42f029e5-9d90-4e67-81ba-ed390a6d05af","Type":"ContainerStarted","Data":"81ceac15144450fddf43b924ffb94f096d4a7c09f595198376fb869b77ead65d"}
Nov 02 23:02:04.978817 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:04.978458    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:05.088355 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:05.088300    5067 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8" podStartSLOduration=2.088284131 podStartE2EDuration="2.088284131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 23:02:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 23:02:05.086856235 +0000 UTC m=+3397.370665924" watchObservedRunningTime="2026-11-02 23:02:05.088284131 +0000 UTC m=+3397.372093817"
Nov 02 23:02:07.579218 ip-10-0-130-41 kubenswrapper[5067]: E1102 23:02:07.579179    5067 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fdcvz" containerName="extractor"
Nov 02 23:02:10.729707 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:10.729672    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-rjgqf_8759d3be-8ddd-4719-808f-3611fcd7f887/cluster-monitoring-operator/0.log"
Nov 02 23:02:10.994602 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:10.994519    5067 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-drfxk/perf-node-gather-daemonset-4ngt8"
Nov 02 23:02:13.326404 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:13.326352    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ljgp_4c9ece87-d259-49f9-bad1-f9573fe2ebf1/node-exporter/0.log"
Nov 02 23:02:13.752192 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:13.752159    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ljgp_4c9ece87-d259-49f9-bad1-f9573fe2ebf1/kube-rbac-proxy/0.log"
Nov 02 23:02:14.177570 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:14.177495    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6ljgp_4c9ece87-d259-49f9-bad1-f9573fe2ebf1/init-textfile/0.log"
Nov 02 23:02:22.152765 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:22.152735    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-2rgrg_0cbcf6f0-083c-43f2-bb04-bab241e8f748/prometheus-operator-admission-webhook/0.log"
Nov 02 23:02:27.666163 ip-10-0-130-41 kubenswrapper[5067]: I1102 23:02:27.666133    5067 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-67c76445c-rm6gt_ec35ca20-d658-4f8a-8725-4d5cd3b5ad87/istio-proxy/0.log"